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EXECUTIVE  SUMMARY 


Internet  Service  Providers  (ISPs)  offer  access  to  the  Internet  and  to  other  network 
resources  to  their  customers.  The  ISP  marketplace  is  extremely  competitive,  requiring 
ISPs  to  provide  their  services  with  limited  resources.  We  develop  models  that  highlight 
tensions  between  changing  customer  demand  and  infrastructure  costs.  The  insights  from 
these  models  can  help  to  improve  ISP  decision  making,  as  well  as  help  to  explain 
ongoing  network  growth. 

We  use  constrained  optimization  to  model  decisions  made  by  ISPs  in  the  face  of 
changing  environmental  conditions.  Specifically,  we  examine  the  traffic  routing 
decisions,  the  investment  in  augmenting  capacity  decisions,  and  building  network 
infrastructure  decisions  made  by  ISPs.  Using  these  models,  we  conduct  numerical 
experiments  to  compare  performance  and  topologies  that  result  from  ISP  decisions. 

We  develop  three  ISP  decision  models:  a  traffic  engineering  model,  a  network 
provisioning  model  and  a  multi-period  network  provisioning  model.  These  models 
capture  the  decisions  ISPs  make  to  satisfy  demand  for  network  traffic  from  customers. 
The  traffic  engineering  model  assumes  the  ISP  can  only  use  the  current  network  to  meet 
demand,  while  the  network  provisioning  models  assume  the  ISP  has  a  budget  to  invest  in 
additional  arc  capacity  and/or  to  build  new  arcs.  We  demonstrate  each  model  with 
simple  examples  that  highlight  tensions  faced  by  ISPs. 

We  use  a  real  ISP,  the  Abilene  network,  as  a  case  study  for  our  decision  models. 
We  introduce  hypothetical  changes  in  demand  to  Abilene,  solve  the  resulting  traffic 
engineering  model,  and  note  the  effects  on  arc  utilization.  Using  an  assumed  budget  and 
the  network  provisioning  model,  we  propose  an  “optimal”  Abilene  built  from  scratch. 
We  compare  the  performance  of  the  “optimal”  Abilene  design  with  the  Abilene  network 
to  elicit  the  factors  network  designers  consider  when  building  network  topologies. 

In  our  numerical  experiments,  we  consider  the  performance  and  network  topology 
of  three  network  design  techniques  on  a  set  of  20  synthetic  demand  matrices.  We 
develop  a  heuristic  design  technique  that  is  reactive  to  changing  customer  demands  and 
augments  capacity  where  arcs  reach  saturation.  We  develop  a  myopic  design  technique 

xii 


that  has  knowledge  of  current  customer  demands  and  invest  optimally  for  the  current 
period.  We  develop  an  omniscient  design  technique  that  has  knowledge  of  customer 
demands  in  all  time  periods  and  invests  optimally  across  all  time  periods.  The  results 
show  that  ISPs  can  improve  performance  either  by  building  network  infrastructure  or  by 
improving  knowledge  of  future  customer  demands. 

We  conduct  a  sensitivity  analysis  by  varying  budget  and  costs.  Intuitively,  an 
increase  in  budget  allows  an  ISP  to  increase  performance;  likewise,  a  decrease  in  costs 
allows  an  ISP  to  increase  performance.  The  results  highlight  an  ISPs  ability  to  improve 
performance  with  either  increased  resources  or  reducing  infrastructure  costs.  The 
network  topologies  result  in  mesh  designs  as  ISPs  invest  to  improve  performance. 

We  vary  the  investment  horizon  in  our  experiments  and  analyze  both  the 
omniscient  and  myopic  design  techniques.  For  the  omniscient  design  technique,  we  show 
that  reducing  an  ISP’s  investment  horizon  deteriorates  performance.  If  an  ISP  has 
knowledge  of  future  customer  demands,  it  is  better  off  investing  resources  as  soon  as 
possible  to  improve  performance  in  current  and  future  periods.  These  results  highlight 
the  value  of  knowing  future  customer  demands. 

For  the  myopic  design  technique,  we  show  that  increasing  an  ISP’s  investment 
horizons  can  improve  performance.  Increasing  investment  horizons  can  provide  two 
benefits  to  an  ISP  operating  under  myopic  design  conditions.  First,  the  ISP  is  able  to 
realize  customer  demands  before  investing  in  augmenting  capacity  and/or  building  new 
arcs.  Second,  the  ISP  has  access  more  resources  to  invest  in  these  realized  customer 
demands.  Although  increasing  an  ISP’s  investment  horizon  offers  these  benefits,  the  ISP 
must  find  a  balance  between  realizing  customer  demands  and  investing  in  these  demands. 
The  results  show  that  choosing  long  investment  horizons  decreases  performance. 

ISPs  can  approach  omniscient  performance  by  choosing  an  investment  horizon 
that  balances  when  and  how  much  to  invest.  The  best  myopic  strategy  seems  to  be  one 
that  waits  long  enough  between  investments  to  observe  real  changes  in  customer 
demands,  but  not  so  long  that  the  network  performs  too  poorly  in  the  meantime.  These 
findings  are  useful  because  ISPs  typically  do  not  know  future  customer  demands  and  can 
implement  policies  that  mimic  the  myopic  technique  discussed  in  this  thesis. 
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I. 


INTRODUCTION 


A.  BACKGROUND 

During  the  last  two  decades,  the  Internet  has  grown  from  a  small  research  network 
to  a  global  communication  infrastructure  that  is  critical  for  both  civilian  and  military 
systems.  This  growing  importance  has  created  considerable  interest  in  the  Internet’s 
structural  features  (e.g.,  efficiency  or  vulnerability)  and  the  way  in  which  these  features 
evolve  over  time.  Both  technologists  and  policy  makers  would  like  to  know,  What  will 
the  future  network  look  like?  and,  What  are  the  drivers  of  this  change?  Part  of  the 
answer  can  be  traced  to  the  owners  and  operators  of  the  Internet  itself. 

The  public  Internet  of  today  is  a  loose  federation  of  more  than  10,000 
independent,  but  interconnected,  communication  networks,  each  known  as  an 
Autonomous  System  (AS).  Each  AS  operates  a  network  comprised  of  routers  and 
communication  links.  A  router  is  a  hardware  device  that  inspects  incoming  data  packets 
and  forwards  each  one  over  an  outbound  link  toward  its  respective  destination.  The 
Internet  Protocol  (IP)  specifies  the  rules  used  by  the  routers  for  forwarding  packets. 

The  owner  and  operator  of  each  AS  is  an  Internet  Service  Provider  (ISP) — a 
business  entity  that  invests  limited  resources  in  the  provisioning  and  management  of  the 
network  in  order  to  achieve  stated  performance  objectives.  An  ISP  can  manage  one  or 
more  independent  networks,  but  without  loss  of  generality,  we  will  assume  that  there  is  a 
one-to-one  correspondence  between  an  AS  and  its  ISP,  often  referring  simply  to  the  ISP. 

Each  AS  faces  a  demand  for  traffic  across  its  network,  typically  characterized  by 
a  matrix  of  origin-destination  (O-D)  pairs,  called  a  demand  matrix.  An  explicit  objective 
of  the  ISP  is  to  satisfy  the  demand  for  traffic  using  limited  network  resources  (e.g.,  link 
bandwidths),  and  often  with  the  objective  of  minimizing  its  operating  cost.  This  is 
challenging  because  the  ISP  does  not  know  the  demand  matrix  in  advance  (although  it 
can  measure  the  current  traffic  on  its  network  at  any  point),  and  this  demand  fluctuates 
over  time. 
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The  ISP  has  two  primary  mechanisms  that  it  can  use  to  influence  whether  or  not 
its  network  will  satisfy  its  demand  matrix.  The  first  is  traffic  engineering — the  ISP  can 
configure  the  routers  and  their  protocols  to  control  the  pathways  used  by  the  data  packets 
as  they  cross  the  network.  The  second  is  network  provisioning — the  ISP  can  invest  to 
increase  the  quantity,  capacity,  and/or  location  of  physical  hardware  in  the  network. 
Network  provisioning  is  sometimes  also  called  network  augmentation  or  network  design, 
depending  on  whether  the  network  of  interest  already  exists  or  is  built  from  scratch. 

This  thesis  focuses  on  the  drivers  of  ISP  network  evolution,  as  reflected  by  the 
traffic  engineering  and  network  provisioning  decisions  of  the  ISP  in  the  face  of  changing 
traffic  demand.  Specifically,  we  formulate  and  solve  constrained  optimization  problems 
representing  these  decision  problems  over  a  multi-period  time  horizon.  These  problems 
are  important  because  the  ISP  marketplace  is  hypercompetitive,  thus  requiring  ISPs  to  do 
more  with  fewer  resources.  In  addition,  constantly  changing  demands  require  ISPs  to 
adapt  their  network  infrastructure  in  an  ongoing  manner.  We  develop  models  that 
highlight  tensions  between  changing  customer  demand  and  infrastructure  costs.  The 
insights  from  these  models  help  to  improve  ISP  decision  making,  as  well  as  help  to 
explain  ongoing  network  growth. 

A  secondary  objective  of  this  thesis  is  to  explore  the  relationship  between  traffic 
demand,  operational  constraints,  and  the  structure  of  the  network  topologies  that  arise 
from  this  design  process.  There  has  been  considerable  academic  attention  on  the  large- 
scale  structure  of  Internet  topologies,  but  most  of  this  work  has  been  descriptive,  rather 
than  explanatory.  Our  aim  is  to  look  at  ISP  network  evolution  form  a  “first  principles” 
perspective  and  characterize  the  network  organization  that  results  from  functional 
requirements. 

B.  LITERATURE  REVIEW  OF  PREVIOUS  WORK 

Our  modeling  and  analysis,  to  explain  the  traffic  engineering  and  network 
provisioning  decisions  made  by  ISPs  over  multi-period  time  horizons,  draws  from 
previous  work  in  the  literature.  We  highlight  several  contributions  and  include  references 
for  a  more  in-depth  coverage  of  the  topics. 
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1.  Traffic  Engineering 

In  practice,  IP  routing  controls  the  flow  of  traffic  based  on  a  network’s  topology 
and  the  configuration  of  protocol  parameters.  The  protocols  obtain  a  set  of  traffic  routing 
paths  without  regard  to  the  traffic  demands  or  the  utilization  of  the  arcs  in  the  network.  It 
is  the  ISP’s  responsibility  to  set  protocol  parameters  such  that  the  ISP’s  customers 
receive  an  acceptable  level  of  service.  Rexford  (2006)  offers  a  model  that  finds  a  set  of 
protocol  parameters  using  optimization.  The  model  uses  existing  network  topology  and 
traffic  demands  as  inputs  to  find  an  optimal  set  of  protocol  parameters  based  on 
performance  objectives  (Rexford,  2006). 

The  traffic  engineering  solutions  we  obtain  using  constrained  optimization  result 
from  traffic  demand  as  input  and  a  specified  performance  measure  (e.g.,  minimize  cost). 
These  solutions  represent  idealized  traffic  engineering,  which  might  not  be  realized  in 
practice  by  actual  routing  protocols,  but  they  are  useful  in  identifying  an  upper  bound  on 
ISP  performance. 

2.  Network  Provisioning  and  Design 

Modeling  an  ISP’s  design  and  provisioning  decisions  using  constrained 
optimization  is  not  new.  Forsgren  and  Prytz  (2006),  and  references  therein,  cover  a  wide 
range  of  network  design  problems  and  their  solutions  using  optimization  models.  The 
performance  criteria  used  in  their  models  include  throughput,  redundancy,  and  cost,  with 
some  discussion  of  the  issues  that  arise  when  choosing  one  performance  criterion  versus 
another. 

In  this  thesis,  we  focus  exclusively  on  cost  as  the  performance  measure  that  drives 
decisions  made  by  ISPs  in  the  provisioning  of  their  networks. 

3.  Network  Topology 

Because  of  the  tremendous  growth  of  the  Internet  over  the  last  20  years, 
considerable  effort  has  gone  to  understanding  its  overall  connectivity.  Much  of  this  effort 
has  been  statistical  in  nature,  with  researchers  emphasizing  features  such  as  the 
distributions  and  correlations  of  node  connectivity. 
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In  contrast  to  the  purely  statistical  characterizations  of  Internet  connectivity,  Li, 
Alderson,  Doyle,  and  Willinger  (2004)  develop  a  “first  principles  approach  to 
understanding  Internet  topology  at  the  router  level”  (p.  1).  The  researchers  develop  a 
model  that  incorporates  functional  requirements,  physical  constraints,  and  economic 
constraints,  which  they  claim  are  insightful  and  reflective  of  network  engineering.  The 
physical  constraints  include  the  number  of  link  connections  possible  using  current  router 
technologies.  The  economic  constraints  include  the  cost  of  infrastructure  such  as  creating 
new  link  connections  between  routers.  They  generate  simply  toy  networks  according  to 
these  constraints  and  compare  their  performance  to  networks  generated  to  match 
observed  statistics.  The  research  shows  that  networks  generated  solely  to  match 
connectivity  statistics  do  not  perform  well  enough  to  be  realistic  representations  of  the 
Internet. 

Alderson,  Li,  Willinger,  and  Doyle  (2005)  use  data  from  real  ISP  networks  to 
provide  experimental  confirmation  of  this  theoretical  result.  They  include  data  from  the 
Corporation  for  Education  Network  Initiatives  (CENIC) — a  regional  network  from 
California  that  “acts  as  an  ISP  for  the  state’s  colleges  and  universities”  (Alderson  et  al., 
2005,  p.  1212),  and  data  from  the  Abilene  Network — “the  national  Internet  backbone  for 
higher  education”  (p.  1211).  Their  research  validates  the  claim  that  “technological  and 
economic  forces  are  relevant  and  real  to  the  real  Internet”  (p.  1211). 

Alderson,  Chang,  Roughan,  Uhlig,  and  Willinger  (2006)  discuss  the  relationship 
between  Internet  topology  and  its  associated  traffic.  A  basic  observation  is  that  the 
architecture  of  the  Internet  leads  to  many  different  vantage  points  from  which  to  measure 
and  model  Internet  topology.  Specifically,  the  TCP/IP  protocol  stack  has  a  physical 
layer,  a  data  link  layer,  a  network  layer,  a  transport  layer,  and  an  application  layer.  As  a 
result,  the  researchers  stress  that  “the  meaning  of  network  ‘topology’  and  ‘traffic’ 
depends  directly  on  one’s  choice  of  focus”  (p.  570). 

Recent  work  done  at  the  Naval  Postgraduate  School  considers  other  features  of 
network  topology  that  are  relevant  to  this  study  of  ISP  network  evolution. 
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Barkley  (2008)  identifies  the  router  or  link  attacks  that  cause  the  worst  possible 
disruption  of  traffic  flow  to  an  IP -based  network.  He  formulates  optimization  models 
that  solve  the  traffic-engineering  problem  for  maximum  flow  across  the  network. 
Barkley  then  formulates  an  optimization  model  that  simulates  the  attack  of  an  intelligent 
adversary  who  wants  to  cause  the  worst  possible  disruption  of  traffic  flow  on  the 
network.  That  is,  the  optimization  problem  minimizes  the  maximum  flow.  He  then  uses 
these  models  to  analyze  the  Abilene  network  for  worst-case  disruptions  and  reports  the 
results. 

Derosier  (2008)  compares  heuristic  and  optimization-based  approaches  to  the 
design  of  ISP  network  topologies.  He  first  analyses  an  existing  U.S.  National  Tier-1  ISP 
and  observes  key  design  principles.  Second,  he  develops  heuristic  and  optimization 
models  to  generate  realistic  ISP  networks  based  on  observations  of  the  Tier-1  ISP. 
Finally,  he  compares  the  performance  of  the  networks  created  using  the  heuristic  models 
and  the  optimization  models,  based  on  cost  and  throughput. 

4.  Traffic  Matrices 

Realistic  traffic  matrices  are  critical  for  assessing  how  ISPs  react  to  changes  in 
their  environment.  Since  ISPs  do  not  make  their  traffic  matrices  available  for  public  use, 
creation  of  realistic  traffic  matrices  is  required  for  analysis.  There  is  extensive  study  in 
the  field  of  generating  traffic  matrices. 

Zhang,  Roughan,  Duffield,  and  Greenberg  (2003)  propose  a  method  for  accurate 
computation  of  traffic  matrices  using  link  load  measurements.  Their  work  uses  these  link 
load  measurements,  which  are  readily  available  in  IP  networks  using  Simple  Network 
Management  Protocol  (SNMP),  along  with  network  routing  configuration  information. 
Tomogravity  is  the  name  of  their  method,  which  has  two  steps.  The  first  step  is  to  use  a 
gravity  model  along  with  the  link  load  data  to  generate  an  initial  solution.  The  second 
step  minimizes  the  problem  size  by  using  knowledge  of  the  topology  configuration  and 
network  routing  and  refines  the  initial  solution  using  quadratic  programming. 

Zhang,  Roughan,  Lund,  and  Donoho  (2003)  propose  a  different  method  for 
estimating  traffic  matrices  using  information  theory.  Realizing  that  most  inference 
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problems  are  ill-posed  because  they  involve  many  more  unknowns  than  known  data;  they 
use  “regularization”  on  the  ill-posed  problem  via  an  entropy  penalization  approach. 
Finally,  the  method  uses  convex  optimization  based  algorithms  to  solve  for  traffic 
matrices. 

Roughan  (2005)  introduces  a  simplification  of  an  approach  to  the  synthesis  of 
traffic  matrices  using  gravity  models.  Roughan  shows  that  using  gravity  models  to 
synthesize  Internet  traffic  matrices  is  reasonable.  He  uses  an  exponential  distribution  to 
generate  the  necessary  independent  and  identically  distributed  random  variables.  He  then 
generates  the  traffic  matrix  using  the  gravity  model.  Gravity  models  received  their  name 
from  Newton’s  law  of  gravitation.  The  idea  is  that  the  demand  for  network  traffic 
between  two  cities,  in  an  ISP  network,  is  proportional  to  the  product  of  their  populations. 

In  this  thesis,  we  generate  synthetic  matrices  for  traffic  demand  according  to  a 
gravity  model.  Henceforth,  we  refer  to  these  matrices  as  demand  matrices. 

5.  Network  Growth 

McPherson  (2009)  contrasts  networks  that  grow  randomly  with  those  that  result 
from  design.  Specifically,  she  considers  probabilistic  graph  formation  models,  including 
classic  Erdos-Renyi  models,  geometric  random  graphs,  and  preferential  attachment 
models.  She  compares  their  behavior  with  optimization-based  models  that  deliberately 
grow  network  structure  to  achieve  a  stated  performance  objective.  McPherson  uses  the 
minimization  spanning  tree  problem  as  the  basis  for  this  comparison. 

This  thesis  extends  this  type  of  analysis  to  the  design  of  ISP  networks. 

C.  OVERVIEW  OF  WORK  IN  THIS  THESIS  AND  CHAPTER  OUTLINE 

In  Chapter  II,  we  formulate  three  optimization  models  including  the  traffic 
engineering  model,  network  provisioning  model,  and  a  multi-period  network 
provisioning  model.  Using  a  few  simple  examples,  we  demonstrate  the  tensions  faced  by 
ISPs  in  traffic  routing  and  network  design.  In  Chapter  III,  we  use  an  existing  ISP, 
Abilene,  as  a  case  study.  In  Chapter  IV,  we  perform  numerical  experiments  that  compare 
the  performance  of  three  design  techniques.  In  Chapter  V,  we  discuss  conclusions  and 
offer  topics  for  further  research. 
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II.  INTERNET  SERVICE  PROVIDER  DECISION  MODELS 


A.  MODELING  AN  ISP  NETWORK 

The  primary  driver  of  ISP  traffic  is  the  demand  from  its  customers.  The 
fundamental  problem  faced  by  an  ISP  is  to  find  a  way  to  route  traffic  through  its  network 
that  satisfies  all  demands.  These  traffic  routes  must  also  meet  operational  constraints 
such  as  the  capacity  available  to  carry  the  traffic  on  each  arc. 

ISPs  typically  face  two  objectives  when  routing  traffic:  minimize  cost  and 
minimize  arc  congestion.  Throughout  this  thesis,  we  focus  on  routing  traffic  to  minimize 
cost,  as  defined  below.  This  performance  measure  is  the  objective  of  all  network  models 
in  this  thesis.  We  consider  the  way  in  which  ISPs  use  traffic  engineering  or  network 
provisioning  to  meet  this  performance  measure. 

B.  NOTATION 

Throughout  this  thesis,  we  follow  the  notation  and  conventions  in  Ahuja, 
Magnanti,  and  Orlin  (1993).  Let  A  be  a  set  of  nodes,  each  representing  a  router  in  an 
ISP  network.  We  define  a  path  as  a  sequence  of  nodes  in  a  network  without  repetition. 
Here,  a  node  can  correspond  to  the  origin,  intermediate  point,  or  destination  along  a 

particular  path.  Let  n  =  1,  2,  ...,  |a|  index  set  A.  Aliases  for  n  include  /',  /,  s,  and  t.  The 
set  A  c  Ax  A  represents  the  set  of  arcs  that  connect  nodes  in  a  network.  We  use  (i,  j)  to 
denote  an  arc  from  node  i  to  node  j. 

Let  Xijt  represent  the  quantity  of  traffic  bound  for  destination  t  that  traverses  arc 
(i,  j),  and  let  w,y  represent  the  upper  bound  (or  capacity )  on  traffic  over  arc  (i,  j).  We 
define  the  utilization  of  arc  (i,  j)  to  be  ^  Xijt  /  uVj ,  the  fraction  of  its  capacity  in  use.  We 

t 

say  that  arcs  with  utilization  near  1.0  are  “congested”  and  arcs  with  utilization  equal  to 
1.0  are  “saturated.” 
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c. 


DEMAND  MATRICES 


We  generate  synthetic  demand  matrices  according  to  a  gravity  model.  Let  bst  be 
the  demand  for  traffic  from  origin  s  bound  for  destination  t.  Let  p(i)  be  the  population  of 
city  i  in  millions  of  people.  Let  y,  be  the  ISP  market  penetration  at  city  i.  Then  the 
population  of  ISP  customers  is  y,  p(i).  A  general  formulation  for  the  demand  between 
two  cities  is  bst  =  ys  p(s )  yt  p(t )  ,  where  ys  is  the  market  penetration  at  city  s,  yt  is  the 

market  penetration  at  city  t,  p(s)  is  the  population  at  origin  city  s,  and  p(t)  is  the 
population  at  destination  city  t.  Using  the  population  of  six  cities  as  input  into  the  gravity 
model,  an  example  O-D  demand  matrix  is  in  Table  1. 


City 

Population 

(millions) 

Nl 

1.50 

N2 

0.25 

N3 

0.75 

N4 

0.75 

N5 

1.25 

N6 

2.50 

N1  N2  N3  N4  N5  N6 


-0.83 

0.04 

0.11 

0.11 

0.19 

0.38 

0.04 

-0.17 

0.02 

0.02 

0.03 

0.06 

0.11 

0.02 

-0.47 

0.06 

0.09 

0.19 

0.11 

0.02 

0.06 

-0.47 

0.09 

0.19 

0.19 

0.03 

0.09 

0.09 

-0.72 

0.31 

0.38 

0.06 

0.19 

0.19 

0.31 

-1.12 

Legend: 


B(s,t)  =  Demand  for  Traffic  Gbps 


Table  1.  Six  cities  and  their  populations  (left),  example  O-D  demand  matrix  using  gravity 
model  and  yt  =  0.32  (right).  For  example,  in  the  O-D  demand  matrix  (right),  the 
entry  in  row  Nl,  column  N2  =  0.04  is  the  demand  from  N1  to  N2.  Likewise,  the 
entry  in  row  Nl,  column  Nl  =  -0.83  is  the  negative  sum  of  the  demand  for  traffic 

from  all  other  cities  to  Nl. 


D.  TRAFFIC  ENGINEERING  TO  MINIMIZE  COST 


The  ISP  seeks  traffic  flows  XiJt  that  minimize  cost  and  satisfies  the  demand  for 
network  traffic.  Let  ^represent  the  cost  to  route  a  unit  of  traffic  across  arc  (i,  j). 
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Let  dy  represent  the  distance  of  arc  (i,  j).  Let  the  elastic  variable  ^represent  the 

quantity  of  traffic  originating  at  5  that  cannot  reach  its  destination  t  due  to  insufficient  arc 
capacity  on  the  network.  We  say  that  such  traffic  has  been  “dropped”  by  the  network. 
We  assume  the  ISP  incurs  a  penalty  p  per  unit  of  undelivered  traffic  Wst  and  that  this 
penalty  is  the  same  for  all  traffic.  To  ensure  that  the  network  incurs  this  penalty  only 

when  no  alternate  path  exists,  we  set  p  =|-/vjc,  where  C  =  maxcy  .  Therefore,  the  penalty 

(UM 

per  unit  of  dropped  traffic  is  more  costly  than  the  per-unit  cost  of  routing  traffic  along  the 
most  expensive  path  in  the  network.  Note  that  when  ctJ  =1 ,  the  operating  cost  reduces  to 

the  number  of  arc  segments,  or  “hops”  traversed  by  all  traffic.  In  contrast,  when  ctj  =  d  y 
the  operating  cost  is  Gb-mile  travel  distance. 

We  assume  the  ISP  knows  the  demand  matrix  that  it  faces.  Using  the  available 
network  topology,  the  ISP  seeks  the  flows  that  minimize  cost.  We  formulate  the 
corresponding  traffic  engineering  model  as  follows. 

Sets 

N  Nodes 

A  Directed  Arcs  A  c  Nx  N 

Index  use  [-cardinality] 

n  e  N  Nodes,  an  ordinal  set  (alias  i,  j,  s,  t)  [-tens] 

(i,  j)  e  A  Directed  arcs  from  i  to  j  arc  (i,  j)  [-hundreds] 

Parameters  [units] 

bst  Demand  (<0)  or  supply  ( >  0  )  from  origin  5  bound  for  destination  t, 

where  btt  =  bst ,  V  t  eN  [Gbps] 

S^t 

c  Cost,  to  send  a  unit  of  traffic  over  arc  (/',  /)  e  A  [cost] 

«,y  Upper  bound  capacity  on  traffic  over  directed  arc  (/',  /)  e  A  [Gbps] 
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p 


Penalty,  per  unit  of  traffic  that  does  not  reach  its  destination 
[cost/Gbps] 


Decision  Variables  [units] 

Xjjt  Quantity  of  traffic  bound  for  destination  t  on  arc  (/',  /)  e  A  [Gbps] 

Wst  Quantity  of  O-D  traffic  from  s  to  t  that  is  dropped  [Gbps] 

Formulation  TE 


min 

x,w 


s.t. 


f  \ 


hN 

II 

M 

+  PXWS, 

(TEO) 

O'JM 

KteN  J 

s,teN 

I  -v 

1 

M 

Os' 

+  Wst  =  V , 

\/s,t  e  N,s 

(TE1) 

j:(s,j)eA 

i:(i,s)eA 

M 

i 

■  X  x« 

~Xw.<=b« 

yteN 

(TE2) 

j:(t,j)eA 

S^t 

V(/J)e 

A 

(TE3) 

t&N 

IV 

© 

V(/,y)e 

< 

m 

(TE4) 

wst  >  o, 

< 

m 

N 

(TE5) 

Discussion 

The  objective  function  (TEO)  includes  the  cost  of  routing  traffic  over  the  network 
and  incurs  a  penalty/?  for  each  unit  of  dropped  traffic  Wst.  The  constraints  (TE1)  and 

(TE2)  ensure  a  balance  of  traffic  flow  at  every  node,  whether  it  is  a  destination  or  not. 
The  constraint  (TE3)  ensures  that  the  traffic  along  arc  (i,  j)  does  not  exceed  the  available 
capacity.  The  constraint  (TE4)  ensures  network  flows  are  nonnegative.  The  constraint 
(TE5)  ensures  that  any  dropped  traffic  is  nonnegative. 
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TE  Example 

With  an  example,  we  demonstrate  the  formulation  and  solution  of  model  TE. 
Consider  a  simple  ISP  network  with  6  nodes  and  16  arcs,  each  arc  with  a  capacity  of  1 
unit.  Assume  Cy  =  1.  This  simple  ISP  network  appears  in  Figure  1. 


Nl. 


JTode  Legend: 


m.o 
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Figure  1.  Simple  ISP  network 

Assume  a  simplified  demand  of  1  unit  from  node  N3  to  node  N4  and  1  unit  from 
node  Nl  to  node  N6.  The  O-D  demand  matrix  appears  in  Table  2. 
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N1  N2  N3  N4  N5  NO 


N1 

N2 

N3 

N4 

N5 

NO 


Table  2.  Simple  ISP  0-D  demand  matrix 

Using  the  known  demand  matrix,  the  ISP  solves  TE  with  ZTe  =  5.  The  flows 
include  1  unit  from  node  N1  to  node  N6,  along  the  path  N1-N3-N5-N6,  and  1  unit  from 
node  N3  to  node  N4  along  the  path  N3-N2-N4.  These  flows  appear  in  Figure  2. 


1.00 

1.00 

-1.00 

-1.00 

Legend: 


B(s,t)  B(s,t)  =  Demand  forTraffic  Gbps 


Figure  2.  Simple  ISP  network  with  solution  to  TE,  ZTe  =  5. 
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E. 


NETWORK  PROVISIONING  TO  MINIMIZE  COST 


In  this  problem,  the  ISP  seeks  traffic  flows  Xit  that  minimize  cost  and  satisfies 

the  demand  for  network  traffic;  however,  the  ISP  has  a  limited  budget  to  invest  in 
additional  capacity  for  the  existing  arcs  and/or  to  build  additional  arcs.  We  assume  that 
building  a  new  arc  incurs  a  fixed  cost,  plus  a  variable  cost  that  increases  with  the  capacity 
of  the  arc,  both  of  which  increase  with  the  physical  length  of  the  arc.  In  contrast,  adding 
capacity  to  an  existing  arc  incurs  only  the  variable  cost. 

We  assume  the  ISP  knows  the  demand  matrix  and  uses  the  available  budget  for 
additional  capacity  to  provision  on  existing  arcs  and/or  to  build  new  arcs  in  the  network. 
The  ISP  seeks  the  best  provisioning  of  resources  and  routing  of  traffic  to  minimize  cost. 
This  formulation  builds  on  the  traffic  engineering  model  with  additional  parameters, 
variables,  and  constraints.  We  formulate  the  network  provisioning  model  as  follows. 

Additional  Parameters  [units] 
dy  Distance  of  arc  (/',  /)  e  A  [miles] 

a  Per  distance  cost  of  building  a  new  arc  [cost/mile] 

/?  Per  unit  distance  cost  of  provisioning  capacity  over  an  arc 

[cost/Gbps-mile] 

maxcapacity  The  maximum  allowable  capacity  to  provision  on  any  new  arc 
(hj)  e  A  [Gbps] 

budget  Total  available  budget  [cost] 
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Additional  Decision  Variables  [units] 


Vg  Binary  Variable  {0,  1},  1  if  arc (/',  /)  e  A  is  built,  and  0  otherwise 

[binary] 

Rjj  Binary  Variable  {0,  1},  1  if  arc (/',  /)  e  A  exists,  and  0  otherwise 

[binary] 

Yy  Quantity  of  additional  capacity  to  provision  on  arc  (i,  j)  e  A 

[Gbps] 

Formulation  NP 


min 

R,V  ,Y  ,X  ,W 


^NP 


z 

(Uj)eA 


\te.N 


ijt 


+pZ  w„ 


s.teN 


(NPO) 


s.t.  (TE1),  (TE2),  (TE4),  (TE5) 


X  Xijt  -  uij  +  Yij  ’ 
t<=N 

(NP1) 

«  Z  d.lYj+R  Z  duyij 

<  budget 

(NP2) 

Yy  <  Ry  maxcapacity, 

W,j)eA 

(NP3) 

V-  <R 

V  ~  '/  ’ 

V(i,y)  e  A 

(NP4) 

+ 

3 

VI 

V(i,/)ed 

(NP5) 

j 

Vie  V 

(NP6) 

ViJ=VJi ’ 

V(i,y)e  A 

(NP7) 

=  ^/'  ’ 

V(/J)eA 

(NP8) 

EV  ={0,1}, 

V(i,/)ed 

(NP9) 

={0,1}, 

V(/,y)e  A 

(NP10) 

IV 

© 

V(i,j)eA 

(NP1 1) 
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Discussion 


As  with  formulation  TE,  the  objective  function  (NPO)  includes  the  cost  of  routing 
traffic  over  the  network  and  incurs  a  penalty  p  for  each  unit  of  dropped  traffic  Wst .  We 

again  have  constraints  (TE1)  and  (TE2)  to  maintain  balance  of  flow  and  constraints, 
along  with  (TE4)  and  (TE5)  to  prevent  nonnegative  flows  and  dropped  traffic.  The 
constraint  (NP1)  ensures  traffic  along  arc  (i,  j)  does  not  exceed  the  available  capacity 

which  now  includes  newly  added  capacity  Yy .  The  constraint  (NP2)  ensures  the  total 

cost  of  additional  network  provisioning  does  not  exceed  the  available  budget.  The 
constraint  (NP3)  ensures  additional  capacity  provisioned  on  arc  (i,  j)  does  not  exceed  a 
maximum  allowable  capacity  limit.  The  constraint  (NP4)  ensures  that  if  arc  (i,  j)  is  built, 
then  it  is  available.  The  constraint  (NP5)  ensures  that  if  arc  (i,  j)  does  not  already  exist 
(i.e.,  uy  =  0)  and  it  is  not  built  (i.e.,  Vy  =  0),  then  it  is  not  available.  In  addition,  we 
prevent  the  unnecessary  construction  of  an  arc  (i,  j)  that  already  exists  by  setting  Vy  =  0  if 
uy  >  0.  The  constraint  (NP6)  ensures  that  for  each  node  an  arc  exists  to  at  least  two  other 
nodes.  The  constraints  (NP7)  and  (NP8)  ensure  symmetry  between  built  arcs.  The 

constraints  (NP9)  and  (NP10)  ensure  Vy  and  Ry  are  binary  variables.  The  constraint 

(NP1 1)  ensures  additional  arc  capacity  on  any  arc  in  the  network  is  nonnegative. 
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NP  Example 

With  a  simple  example,  we  demonstrate  the  formulation  and  solution  to  model 
NP.  Consider  a  simple  ISP  network  with  6  nodes  and  16  arcs,  each  arc  with  a  capacity  of 
1  unit.  Assume  the  ISP  has  the  option  to  build  and  provision  arcs  (N2,  N5),  (N5,  N2), 
(N3,  N4),  and/or  arc  (N4,  N3)  which  appear  as  dashed  lines.  Assume  cy  =  1  and  dy  =  1 
and  that  the  ISP  faces  the  demand  matrix  in  Table  2.  This  simple  ISP  network  is  in 
Figure  3. 


Wo 


N1 


4k 


m.o 


£Tode  Legend: 


Wo 


-N6 
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Arc  Legend:  . 

Capacity,  Utilization 

Utilization  Scale 
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Figure  3.  Simple  ISP  network  with  4  potential  arcs. 

Assume  the  ISP  has  a  budget  of  3  units  to  augment  capacity  and/or  build  arcs  and 
that  parameter  a  =  1  and  /?  =  1.  The  ISP  solves  NP  using  the  budget  and  the  demand 
matrix  in  Table  2.  After  solving  NP,  ZNp  =  4.  The  ISP  invests  2  units  to  build  arc  (N3, 
N4)  and  (N4,  N3),  and  1  unit  to  provision  1  unit  of  capacity  on  arc  (N3,  N4).  The  flows 
include  1  unit  from  node  N1  to  node  N6,  along  the  path  N1-N2-N4-N6,  and  1  unit  from 
node  N3  to  node  N4  along  the  newly  built  arc  (N3,  N4).  The  resulting  network  is  in 
Figure  4. 
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Figure  4.  Simple  ISP  network  with  additional  arc  (N3,  N4)  built  and  provisioned  with  1 

unit  of  capacity;  ZNP  =  4. 

F.  MULTI-PERIOD  NETWORK  PROVISIONING  TO  MINIMIZE  COST 

We  now  consider  the  network  provisioning  model  extended  for  multiple  time 
periods  and  where  the  ISP  has  an  operating  budget  in  each  time  period  to  invest  in 
additional  capacity  for  the  existing  arcs  and/or  to  build  additional  arcs.  By  construction, 
the  ISP  cannot  provision  an  arc  unless  it  is  built.  We  assume  that  an  arc,  once  built 
and/or  provisioned  with  new  capacity,  is  available  for  all  future  time  periods.  We  let  T 
represent  the  set  of  periods  over  which  we  are  solving  the  network  design  model,  and 
use  ref  to  denote  each  time  period. 

As  before,  we  assume  the  ISP  knows  the  values  in  the  demand  matrix  for  all 
periods.  The  formulation  is  similar  to  the  network  provisioning  model  with  the  addition 
of  a  set  of  periods  and  additional  parameters,  variables,  and  constraints.  We  formulate 
the  multi-period  network  design  model  for  finding  traffic  flows  that  meet  demand  at 
minimum  cost. 
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Additional  Sets 


T  Periods 

Additional  Index  use  [-cardinality] 

r  e  T  Periods  [-tens]  alias  f 

Additional  Parameters  [units] 

b]t  Demand  (<0)  or  supply  ( >  0  )  from  origin  s  bound  for  destination  t 

in  period  r  where  b'n  =  [Gbps] 

S^t 

budget T  Total  available  budget  in  period  r  [cost] 

Decision  Variables  [units] 

BT  The  quantity  of  budget  to  spend  in  period  r  [cost] 

Rjj  Binary  Variable  {0,  1},  1  if  arc (i,j)eA  exists  in  period  r  0 

otherwise  [binary] 

V-j  Binary  Variable  {0,  1},  1  if  arc (/',  /)  e  A  is  built  in  period  r  0 

otherwise  [binary] 

Yy  Quantity  of  additional  capacity  to  provision  on  arc  (i,  j)  g  A  in 

period  r  [Gbps] 

X]-t  Quantity  of  traffic  bound  for  destination  t  on  arc  ( i,j)eA  in 

period  r  [Gbps] 

W/t  Quantity  of  O-D  traffic  that  is  dropped  in  period  r  [Gbps] 
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Formulation  MPNP 
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Discussion 


The  objective  function  (MPNPO)  minimizes  the  cost  of  routing  traffic  over  the 
network  over  all  periods,  again  with  a  penalty  p  for  each  unit  of  dropped  traffic  W/t .  The 

constraints  (MPNP1)  and  (MPNP2)  ensure  a  balance  of  traffic  at  every  node  in  each 
period.  The  constraint  (MPNP3)  ensures  the  traffic  along  arc  (i,  j)  does  not  exceed  the 
available  capacity  in  period  r.  The  constraints  (MPNP4)  and  (MPNP5)  ensure  that  the 
cost  of  additional  capacity,  provisioned  on  arc  (i,  j),  and  the  cost  of  additional  arcs,  does 
not  exceed  the  available  budget.  The  constraint  (MPNP6)  ensures  that  if  an  arc  does  not 
exist,  then  it  can  build  at  most  once.  The  constraint  (MPNP7)  ensures  that  if  the  arc 
exists,  then  it  cannot  be  built  in  any  future  period.  The  constraints  (MPNP8),  (MPNP9), 
and  (MPNP10)  ensure  that  if  we  build  an  arc  in  period  r,  then  it  exists  in  all  future 
periods.  The  constraint  (MPNP1 1)  ensures  that  the  additional  capacity  added  to  arc  (i,  j) 
in  period  r  does  not  exceed  a  maximum  allowable  capacity  limit.  The  constraint 
(MPNP12)  ensures  that  if  an  arc  already  exists  before  the  first  period,  then  that  arc  will 
exist  for  all  periods.  The  constraint  (MPNP13)  ensures  that  we  can  build  an  arc  in  future 
periods  if  it  does  not  exist  before  period  one.  The  constraint  (MPNP14)  ensures  that  for 
each  node,  an  arc  exists  to  at  least  two  other  nodes.  The  constraints  (MPNP15)  and 
(MPNP16)  ensure  that  symmetry  exists  between  built  arcs.  The  constraints  (MPNP17) 

and  (MPNP18)  ensure  that  V?  and  Rjj  are  binary  variables.  The  constraint  (MPNP19) 

ensures  that  dropped  flows  are  non-negative.  The  constraint  (MPNP20)  ensures  no 
negative  traffic  on  any  arc  in  the  network.  The  constraint  (MPNP21)  ensures  no  negative 
additional  arc  capacity  on  any  arc  in  the  network. 
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MPNP  Example:  The  Value  of  Knowing  Future  Demands 

In  this  example,  we  demonstrate  use  of  the  MPNP  model  and  the  value  of 
knowing  future  demand.  Again,  consider  the  ISP  network  in  Figure  3.  As  before,  we 
assume  that  the  ISP  is  considering  building  and  provisioning  arcs  (N2,  N5),  (N5,  N2), 
(N3,  N4),  and/or  arc  (N4,  N3),  which  appear  as  dashed  lines. 

We  consider  two  periods,  x  =  1  and  x  =  2.  We  assume  the  ISP  has  a  budget,  B  =  3, 
to  invest  in  additional  arc  capacity  and/or  building  new  arcs  and  that  parameters  a  =  1 
and  /?  =  1 . 

Without  Knowledge  of  Future  Demands 

For  x=  1,  the  ISP  solves  NP  using  B  and  the  demand  matrix  in  Table  2.  The  ISP 
invests  2  units  to  build  arc  (N3,  N4)  and  (N4,  N3),  and  1  unit  to  increase  the  capacity  of 
arc  (N3,  N4)  to  1  unit.  ZNP  =  4.  As  before,  the  resulting  network  is  Figure  4. 

At  x  =  2,  assume  the  demand  for  network  traffic  changes,  increasing  demand  from 
node  N2  to  node  N4  by  2  units.  The  r  =  1  and  x  =  2  demand  matrices  appear  in  Table  3. 


N1  N2  N3  N4  N5  N6 
N1 

N2 

N3 

N4 

N5 

N6 


1.00 

1.00 

-1.00 

-1.00 

N1  N2  N3  N4  N5  N6 
N1 

N2 

N3 

N4 

N5 

N6 


1.00 

2.00 

1.00 

-3.00 

-1.00 

Legend: 

B(s.t) 


B(s,t)  =  Demand  forTraffic  Gbps 


Legend: 

B(s.t) 


B(s,t)  =  Demand  forTraffic  Gbps 


Table  3.  Demand  matrix  at  x  =  2  (left),  demand  matrix  at  x  =  2  (right). 
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Since  the  ISP  used  its  entire  budget  B  in  z  =  1,  the  ISP  solves  TE  with  new 
demands,  ZTe  =  1 1  at  z  =  2.  The  flows  include  1  unit  from  node  N2  to  node  N4,  along  the 
arc  (N2,  N4),  1  unit  from  node  N2  to  node  N4  along  path  N2-N3-N5-N4,  and  1  unit  from 
node  N3  to  node  N4  along  arc  (N3,  N4).  The  network  at  x  =  2  is  in  Figure  5.  Note  that 
one  unit  of  traffic  destined  for  node  N6  is  dropped,  as  illustrated  in  Table  4. 


Figure  5.  Without  knowledge  of  future  demand,  ZNP  =  11  at  z  =2. 
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0  2  4  6  a  10  12  14 


N1  N2  N3  N4  N5  N6 


Table  4. 


N1  |;8 
N2  |;8 
N3  |;8 

N4  1q  |30 
N5  |;8 
N6 

Legend: 

J  Total  Delivered  Traffic  in  Gbps 
|  Total  Dropped  Traffic  in  Gbps 


N1 

N2 

N3 

N4 

N5 

N6 


1.00 

1.00 

2.00 

1.00 

-3.00 

-1.00 

Legend: 

B(s.t) 


B(s,t)  =  Demand  forTraffic  Gbps 


B(s,t)  B(s,t)  =  Demand  forTraffic  Gbps 
W(s,t)  W(s,t)  =  Dropped  Traffic  Gbps 


Delivered  and  dropped  traffic  (left),  demand  matrix  at  r  =  2  (right).  For  example, 
in  the  demand  matrix  on  the  right,  the  shaded  cell  at  row  Nl,  column  N6, 
represents  dropped  demand.  The  top  number  represents  the  demand,  while  the 
bottom  number  represents  the  quantity  of  that  demand  that  is  dropped. 


With  Knowledge  of  Future  Demands 


Again,  consider  the  ISP  network  in  Figure  3.  Assume  the  ISP  knows  the  demand 
matrices  for  both  x  =  1  and  x  =  2  from  the  outset.  The  ISP  solves  MPNP  for  r  =  I  and  x  = 
2  with  B  and  both  demand  matrices,  and  obtains,  ZMpnp  =  12;  5  at  x  =  1  and  7  at  r  =  2. 
The  flows  at  x  =  2  include  1  unit  from  node  Nl  to  node  N6,  along  the  path  N1-N3-N5- 
N6,  1  unit  from  node  N3  to  node  N4  along  path  N3-N2-N4  and  2  units  on  arc  (N2,  N4). 
All  demand  is  satisfied.  The  ISP  network  at  x  =  2  is  in  Figure  6.  Note  the  ISP  invests  B 
to  augment  the  capacity  on  arc  (N2,  N4)  to  3  units. 


23 


Figure  6.  With  knowledge  of  future  demand,  ZMpnp  =  12. 

Without  knowledge  of  future  demands  the  ISP  minimizes  costs  at  x  =  1  at  the 
expense  of  future  costs  and  dropped  traffic  at  r  =  2.  The  ISP  solved  NP  at  x  =  1  and  TE 
at  x  =  2  for  a  ZNP  =  4  and  a  ZTe  =  1 1  for  a  total  ZNP  +  ZTe  =15. 

With  knowledge  of  future  demand,  the  ISP  delays  myopic  gains  in  favor  of 
optimal  provisioning  for  both  x  =  1  and  x  =  2.  The  ISP  solved  MPNP  at  x  =  1  and  x  =  2 
for  a  Zmpnp  =12. 

The  value  of  knowledge  in  this  example  is  4  units.  With  knowledge  of  future 
demands,  the  optimal  solution  is  for  the  ISP  to  incur  a  greater  expense  in  x  =1  in  order  to 
achieve  a  greater  savings  in  x  =  2.  These  findings  appear  in  Figure  7. 
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ISP  Performance  Comparison 
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Figure  7.  A  comparison  of  performance  with  and  without  knowledge  of  future  demand. 
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III.  CASE  STUDY:  ABILENE 


A.  THE  ABILENE  NETWORK 

Abilene  is  a  high-speed  research  and  education  network  providing  IP  services  to 
universities  throughout  the  United  States  (Qwest,  2010).  Internet2,  Cisco,  Nortel 
Networks,  Juniper  Networks,  Qwest  Communications,  and  Indiana  University  created 
Abilene  in  a  joint  effort  (Qwest,  2010).  The  network  is  used  by  hundreds  of  universities 
in  the  United  States  for  services  such  as  “tele-immersion,  virtual  laboratories,  distance 
learning,  distributed  performing  arts,  tele-medicine,  grid  computing  and  digital  libraries” 
(Qwest,  2010). 

Figure  8  illustrates  the  topology  of  Abilene  (Qwest,  2010).  Abilene  has  Points-of- 
Presence  (POPs)  in  11  cities  throughout  the  U.S.  connected  via  28  arcs,  each  with  10 
Gbps  of  capacity. 


The  Abilene  Network 
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Figure  8.  Abilene  topology  (From  Qwest,  2010). 

Although  the  topology  of  Abilene  is  well  documented,  the  demand  matrices  are 

unknown.  We  generate  the  synthetic  O-D  demand  matrix  in  Table  6  using  the  gravity 

model  in  Chapter  II  and  the  population  of  each  POP  in  Table  5. 
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Customer 


Population 

POP  Population 

(millions) 

(millions) 


Atlanta  (ATL) 

0.50 

0.16 

Chicago  (CHI) 

2.90 

0.93 

Denver  (DEN) 

0.60 

0.19 

Houston  (HOU) 

2.20 

0.70 

Indianapolis  (IND) 

0.80 

0.26 

Kansas  City  (KSC) 

0.50 

0.16 

Los  Angeles  (LAX) 

3.80 

1.22 

New  York  (NYC) 

8.40 

2.69 

Sunnyvale  (SUN) 

0.10 

0.03 

Seattle  (SEA) 

0.60 

0.19 

Washington  D.  C.  (WDC) 

0.60 

0.19 

Table  5.  Population  and  assumed  customer  population  of  1 1  Abilene  POPs;  population 

data  from  U.S.  Census  Bureau  (2010). 
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ATL 

CHI 

DEN 

HOU 

IND 

KSC 

LAX 

NYC 

SUN 

SEA 

WDC 

ATL 

-1.03 

0.14 

0.03 

0.11 

0.04 

0.02 

0.19 

0.42 

0.01 

0.03 

0.03 

CHI 

0.14 

-5.25 

0.17 

0.64 

0.23 

0.14 

1.10 

2.44 

0.03 

0.17 

0.17 

DEN 

0.03 

0.17 

-1.22 

0.13 

0.05 

0.03 

0.23 

0.50 

0.01 

0.04 

0.04 

HOU 

0.11 

0.64 

0.13 

-4.14 

0.18 

0.11 

0.84 

1.85 

0.02 

0.13 

0.13 

IND 

0.04 

0.23 

0.05 

0.1S 

-1.62 

0.04 

0.30 

0.67 

0.01 

0.05 

0.05 

KSC 

0.02 

0.14 

0.03 

0.11 

0.04 

-1.03 

0.19 

0.42 

0.01 

0.03 

0.03 

LAX 

0.19 

1.10 

0.23 

0.84 

0.30 

0.19 

-6.54 

3.19 

0.04 

0.23 

0.23 

NYC 

0.42 

2.44 

0.50 

1.85 

0.67 

0.42 

3.19 

-10.58 

0.08 

0.50 

0.50 

SUN 

0.01 

0.03 

0.01 

0.02 

0.01 

0.01 

0.04 

0.08 

-0.21 

0.01 

0.01 

SEA 

0.03 

0.17 

0.04 

0.13 

0.05 

0.03 

0.23 

0.50 

0.01 

-1.22 

0.04 

WDC 

0.03 

0.17 

0.04 

0.13 

0.05 

0.03 

0.23 

0.50 

0.01 

0.04 

-1.22 

Legend: 

B(s,t) 


B(s,t)  =  Demand  for  Traffic  Gbps 


Table  6.  Abilene  O-D  demand  matrix. 

B.  TRAFFIC  ENGINEERING  ON  EXISTING  NETWORK 

Using  the  demand  matrix  in  Table  6  and  the  known  topology  of  Abilene,  we  solve 
model  TE  and  obtain  a  ZTe  of  91.  The  resulting  traffic  flows  appear  in  Figure  9.  The 
delivered  traffic  and  arc  capacities  appear  in  Table  7. 
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Table  7. 


SEA 


Figure  9.  ZTe  =  91  for  Abilene  under  assumed  demand  in  Table  6. 
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Legend: 

_ |  Total  Delivered  Traffic  in  Gbps 


ATL 

CHI 

DEN 
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NYC 
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ATL 

8.13 

10.00 

2.45 

10.00 
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DEN 

10.00 

0.37 

10.00 
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10.00 
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Legend: 


X(s,t)  X(s,t)  =  Arc  Flow,  X(origin.destination)  Gbps 
U(s,t)  U(s,t)  =  Arc  Capacity,  U(origin, destination)  Gbps 

Utilization  Scale 


0.0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1.0 


POP  delivered  traffic  (left),  arc  flow  and  capacity  matrix  (right).  For  example,  in 
the  arc  flow  and  arc  capacity  matrix  on  the  right,  cells  can  contain  two  numbers. 
The  top  number  is  the  total  traffic  flow,  the  bottom  number  is  the  available 

capacity. 
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c. 


EXAMPLES:  HYPOTHETICAL  CHANGES  IN  DEMAND 


1.  Increase  Demand:  LAX  and  NYC 

We  consider  the  case  where  demands  increase  between  LAX  and  NYC  by  10 
Gbps.  The  new  demand  matrix  is  in  Table  8.  We  solve  TE  with  the  new  demand  matrix 
and  note  the  utilization  on  the  arcs  in  the  network.  The  resulting  traffic  flows  are  in 
Table  8.  Note  the  dropped  traffic  between  NYC  and  LAX  in  Table  8. 


ATL 

CHI 

DEN 

HOU 

IND 

KSC 

LAX 

NYC 

SUN 

SEA 

WDC 


ATL  CHI  DEN  HOU  IND  KSC  LAX  NYC  SUN  SEA  WDC 


-1.03 

0.14 

0.03 

0.11 

0.04 

0.02 

0.19 

0.42 

0.01 

0.03 

0.03 

0.14 

-5.25 

0.17 

0.64 

0.23 

0.14 

1.10 

2.44 

0.03 

0.17 

0.17 

0.03 

0.17 

-1.22 

0.13 

0.05 

0.03 

0.23 

0.50 

0.01 

0.04 

0.04 

0.11 

0.64 

0.13 

-4.14 

0.18 

0.11 

0.84 

1.85 

0.02 

0.13 

0.13 

0.04 

0.23 

0.05 

0.18 

-1.62 

0.04 

0.30 

0.67 

0.01 

0.05 
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0.19 
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0.92 
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0.01 

0.01 

0.04 

0.08 

-0.21 

0.01 

0.01 

0.03 

0.17 

0.04 

0.13 

0.05 

0.03 

0.23 

0.50 

0.01 

-1.22 

0.04 

0.03 

0.17 

0.04 

0.13 

0.05 

0.03 

0.23 

0.50 

0.01 

0.04 

-1.22 

Legend: 


B(s,t) 


B(s,t)  =  Demand  for  T raffle  Gbps 


B(s,t) 

W(s,t) 


B(s,t)  =  Demand  forTrafflc  Gbps 
W(s,t)  =  Dropped  Traffic  Gbps 


Table  8.  Demand  between  LAX  and  NYC  increases  by  10  Gbps  from  Table  6. 

Notice  the  saturated  arcs  Figure  10.  In  particular,  note  arc  (WDC,  NYC),  arc 
(NYC,  WDC),  arc  (IND,  CHI),  and  arc  (CHI,  IND).  These  arcs  create  a  cut — a  partition 
of  the  node  set  N  into  two  parts  because  no  additional  traffic  can  flow  between  the  nodes 
in  either  partition.  In  Abilene,  this  cut  puts  SEA,  SUN,  LAX,  DEN,  KSC,  HOU,  IND, 
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ATL,  and  WDC,  in  one  partition,  CHI  and  NYC  in  the  other.  The  cut-set — the  set  of  arcs 
that  partition  the  nodes  in  a  network  into  two  disjoint  sets,  includes  the  four  saturated 
arcs.  The  dropped  traffic  is  a  result  of  the  cut  in  Abilene. 

SEA 

fc 


Figure  10.  Saturated  arcs  result  in  a  cut  on  Abilene. 

In  this  example,  we  demonstrate  one  possible  result  of  increasing  demands 
between  two  POPs  on  a  network.  Without  an  intelligent  ISP  augmenting  capacity  on 
congested  arcs,  they  become  saturated  resulting  in  a  cut.  The  cut  prevents  nodes  in  the 
network  from  communicating  causing  traffic  to  drop.  An  ISP  must  monitor  and  augment 
capacity  on  congested  arcs  to  prevent  dropping  traffic. 

2.  Increase  Demand:  HOU  and  IND 

We  consider  the  case  where  demands  increase  between  HOU  and  IND  by  10 
Gbps.  The  new  demand  matrix  is  in  Table  9.  We  solve  TE  with  the  new  demand  matrix 
and  note  the  utilization  on  the  arcs  in  the  network.  The  resulting  traffic  flows  are  in 
Figure  11.  Note  the  dropped  traffic  between  NYC  and  SUN  and  between  NYC  and  SEA 
in  Table  9. 
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ATL  CHI  DEN  HOU  IND  KSC  LAX  NYC  SUN  SEA  WDC 


ATL 

-1.03 

0.14 

0.03 

0.11 

0.04 

0.02 

0.19 

0.42 

0.01 

0.03 

0.03 

CHI 

0.14 

-5.25 

0.17 

0.64 

0.23 

0.14 

1.10 

2.44 

0.03 

0.17 

0.17 

DEN 

0.03 

0.17 

-1.22 

0.13 

0.05 

0.03 

0.23 

0.50 

0.01 

0.04 

0.04 

HOU 

0.11 

0.64 

0.13 

-14.14 

10.18 

0.11 

0.84 

1.85 

0.02 

0.13 

0.13 

IND 

0.04 

0.23 

0.05 

10.13 

-11.62 

0.04 

0.30 

0.67 

0.01 

0.05 

0.05 

KSC 

0.02 

0.14 

0.03 

0.11 

0.04 

-1.03 

0.19 

0.42 

0.01 

0.03 

0.03 

LAX 

0.19 

1.10 

0.23 

0.84 

0.30 

0.19 

-6.54 

3.19 

0.04 

0.23 

0.23 

NYC 

0.42 

2.44 

0.50 

1.85 

0.67 

0.42 

3.19 

-10.58 

0.08 

0.50 

0.50 

0.08 

0.21 

SUN 

0.01 

0.03 

0.01 

0.02 

0.01 

0.01 

0.04 

0.08 

-0.21 

0.01 

0.01 

0.08 

SEA 

0.03 

0.17 

0.04 

0.13 

0.05 

0.03 

0.23 

0.50 

0.01 

-1.22 

0.04 

0.21 

WDC 

0.03 

0.17 

0.04 

0.13 

0.05 

0.03 

0.23 

0.50 

0.01 

0.04 

-1.22 

Legend: 


W(s,t) 


B(s,t)  =  Demand  for  T raffle  Gbps 


B(s,t)  =  Demand  for  T raffle  Gbps 
W(s,t)  =  Dropped  Traffic  Gbps 


Table  9.  Demand  between  HOU  and  IND  increases  by  10  Gbps  from  Table  6. 

Notice  the  saturated  arcs  Figure  11.  In  particular,  note  arc  (KSC,  IND),  arc  (IND, 
KSC),  arc  (HOU,  ATL),  and  arc  (ATL,  KSC).  These  arcs  create  a  cut  in  Abilene  with 
SEA,  SUN,  LAX,  DEN,  KSC,  and  HOU  in  one  partition,  CHI,  IND,  ATL,  NYC,  and 
WDC  in  the  other.  The  cut-set  includes  the  four  saturated  arcs.  The  dropped  traffic  is  a 
result  of  the  cut  in  Abilene. 
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SEA 


Because  the  network  does  not  have  residual  capacity  to  route  additional  flow  from 
HOU  to  IND,  it  would  seem  that  an  increase  in  demand  between  these  POPs  would  cause 
traffic  to  drop  between  these  two  POPs;  however,  this  is  not  the  case. 

We  increase  the  demand  between  HOU  and  IND  by  5  Gbps.  The  new  demand 
matrix  is  in  Table  10.  We  solve  TE  with  the  new  demand  matrix.  Note  that  no  traffic  is 
dropping  between  HOU  and  IND;  however,  traffic  is  dropping  between  other  POPs  that 
reside  on  either  side  of  the  cut  in  Table  10. 
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ATL 

CHI 

DEN 

HOU 

IND 

KSC 

LAX 

NYC 

SUN 

SEA 

WDC 
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0.02 

0.19 

0.42 
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Legend: 

B(s.t) 


B(s.t) 

W(s,t) 


B(s,t)  =  Demand  forTraffic  Gbps 


B(s,t)  =  Demand  forTraffic  Gbps 
W(s,t)  =  Dropped  Traffic  Gbps 


Table  10.  Demand  between  HOU  and  IND  increases  by  5  Gbps  from  Table  9. 

An  increase  in  traffic  between  POPs  on  either  side  of  the  cut  causes  traffic  to 
drop;  however,  it  is  less  expensive  to  route  traffic  across  shorter  paths  and  drop  traffic 
across  longer  paths. 

For  example,  because  of  the  distance,  routing  1  Gbps  of  traffic  between  LAX  and 
NYC  is  more  expensive  than  routing  1  Gbps  of  traffic  between  HOU  and  IND.  Thus,  if  1 
Gbps  of  traffic  must  be  dropped,  the  minimum  cost  solution  drops  the  most  expensive 
flows  first. 

This  objective  of  minimum  cost  prefers  to  drop  longer  flows  in  exchange  for 
shorter  less  expensive  flows  that  traverse  the  cut.  Once  all  of  these  more  expensive  flows 
drop  any  increase  in  demand  between  HOU  and  IND  will  begin  to  drop. 
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Now  consider  an  increase  in  the  demand  for  traffic  between  IND  and  HOU  of  5 
Gbps.  The  new  demand  matrix  is  in  Table  11.  We  solve  TE  with  the  new  demand 
matrix.  Note  that  traffic  is  now  dropping  between  HOU  and  IND  in  Table  1 1 . 


Legend: 

B(s.t) 


B(s.t) 

W(s,t) 


B(s,t)  =  Demand  forTraffic  Gbps 


B(s,t)  =  Demand  forTraffic  Gbps 
W(s,t)  =  Dropped  Traffic  Gbps 


Table  1 1 .  Demand  between  HOU  and  IND  increases  by  5  Gbps  from  Table  10. 

Thus,  we  observe  that  traffic  routing  is  not  intuitive.  The  objective  of  the  ISP  will 
drive  the  routes  chosen.  In  this  example,  minimizing  cost  drives  traffic  routing  to  prefer 
shorter  routes. 
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D.  THE  “OPTIMAL”  ABILENE  NETWORK 

1.  “Clean  Slate”  Network  Design 

We  consider  a  simple  question:  If  an  ISP  could  build  a  network  from  scratch  to 
serve  the  Abilene  POPs,  what  would  it  look  like? 

We  start  with  the  eleven  cities  corresponding  to  Abilene  POPs  in  the  United 
States.  These  cities  and  their  populations  appear  in  Table  5. 

We  consider  all  (11)  (10)  =110  possible  arcs  between  the  eleven  POPs;  they 
appear  as  dashed  lines  in  Figure  12.  We  generate  a  synthetic  demand  matrix  based  on  the 
gravity  model  discussed  in  Chapter  II.  This  O-D  demand  matrix  is  in  Table  6. 
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Figure  12.  1 10  possible  arcs  in  the  design  of  an  “optimal”  Abilene. 

We  infer  a  budget  from  the  existing  Abilene  topology,  which  includes  28  arcs 
with  10  Gbps  of  capacity  provisioned  on  each  arc.  Let  a  =  10  and  /?  =  1. 
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We  calculate  an  inferred  budget  B  from  the  following  formula: 

budget  =  a  V'  J  ■  ■+ /?  V"  d-  ,\/(i,j)  e  Abilene  . 

(i,j)sA 

We  endow  the  ISP  with  an  initial  budget  B  =  304,466  to  design  and  provision  the 
network  in  a  way  that  minimizes  ZNP. 

The  network  designer  faces  a  choice  when  building  a  network:  build  direct 
connections  between  O-D  pairs  (and  incur  fixed  link  costs),  or  build  fewer  arcs  but 
increase  the  distance  over  which  traffic  travels.  The  relative  merit  of  each  depends  on 
parameters  a  and  /?.  Without  loss  of  generality,  suppose  /?  =  1 .  Small  values  of  a  mean 
that  the  fixed  costs  of  building  an  arc  are  relatively  small.  In  contrast,  when  a  is  large, 
the  fixed  arc  costs  are  expensive.  We  use  the  NP  model,  budget  B,  the  values  of 
parameter  a  and  [i  and  the  demand  matrix  from  Table  6. 

Consider  first  the  case  when  a  =  0.  We  solve  model  NP  with  B  =  304,466,  and 
the  demand  matrix  from  Table  6.  The  resulting  network  design  appears  in  Figure  13. 
From  this,  we  observe  that  when  a  =  0,  the  “optimal”  design  builds  direct  connections  as 
long  as  there  is  sufficient  budget. 
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SEA 


Figure  13.  The  ISP  builds  110  arcs  with  a  =  0  and  routes  traffic  at  ZNP  =  34. 

Next,  we  solve  NP  with  a  =  5,  10,  15,  and  20  and  note  the  results.  By  increasing 
the  value  of  a,  building  additional  arcs  becomes  more  expensive.  For  example,  in  order 
for  the  ISP  to  use  arc  (HOU,  ATL),  dy  =  608  miles,  the  ISP  will  have  to  pay  an  upfront 
cost  of  a  608  prior  to  provisioning  any  capacity  on  that  arc. 

We  solve  NP  with  B  =  304,466,  each  value  of  a,  and  the  demand  matrix  in  Table 
6.  The  resulting  topology  of  four  “optimal”  network  designs  appear  in  Figure  14. 
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Figure  14.  Topology  of  four  “optimal”  Abilene  network  designs. 

Even  when  facing  the  same  demand  matrix,  different  costs  associated  with 
building  network  infrastructure  can  drive  ISPs  to  design  different  network  topologies. 
This  result  appears  in  Figure  14.  As  costs,  controlled  in  this  example  by  a,  to  build  arcs 
increase,  ISPs  build  fewer  arcs  and  route  more  traffic  on  these  arcs.  At  the  same  time,  the 
ZNP  increases.  These  results  appear  in  Figure  15. 
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Performance  (ZNP)and  Arcs  Built 


Figure  15.  “Optimal”  Abilene  performance,  ZNP,  and  number  of  arcs  built  versus  an 

increasing  parameter  a. 
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2.  “Optimal”  Abilene  versus  Abilene 

We  solve  model  NP  with  a  =  15,  an  initial  budget  B  =  304,466  and  the  O-D 
demand  matrix  in  Table  6.  The  resulting  “optimal”  Abilene  design  appears  in  Figure  16. 


SEA 


Figure  16.  “Optimal”  Abilene,  ZNP  =  57. 

We  compare  the  “optimal”  Abilene  design  with  Abilene.  The  “optimal”  Abilene 
design  has  26  arcs  while  Abilene  has  28.  The  “optimal”  Abilene  design  performs  at 
nearly  half  the  ZNP  of  Abilene.  All  arcs  in  the  “optimal”  Abilene  are  saturated  while  the 
maximum  arc  utilization  on  Abilene  is  81%.  A  comparison  of  the  “optimal”  and  real 
Abilene  designs  is  in  Figure  17. 
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While  the  “optimal”  Abilene  outperforms  the  real  Abilene,  the  resulting  topology 
is  “optimal”  only  for  that  demand.  In  the  “optimal”  design,  all  arcs  are  saturated,  thus,  an 
increase  in  demand  will  cause  the  “optimal”  design  to  drop  traffic.  In  contrast,  Abilene  is 
well  equipped  to  handle  an  increase  in  demand.  The  average  arc  utilization  on  the  real 
Abilene  design  is  only  32%,  thus  the  arcs  on  the  real  Abilene  design  are  capable  of 
handling  an  increase  in  demand.  The  penalty  the  real  Abilene  design  incurs  is  poor 
performance  relative  to  the  “optimal”  design.  A  comparison  of  Abilene  with  four 
“optimal”  Abilene  designs  with  a  =  0,  5,  10,  15,  and  20  appears  in  Table  12. 
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Abilene 

“Optimal” 

a  =  0 

“Optimal” 

a  =  5 

“Optimal” 

a  =  10 

“Optimal” 

a  =  15 

“Optimal” 

a  =  20 

Number  of 

Arcs 

28 

110 

58 

34 

26 

24 

Performance 

(Znp) 

91 

34 

37 

44 

57 

79 

Maximum 

Arc 

Utilization 

0.81 

1.00 

1.00 

1.00 

1.00 

1.00 

Minimum 

Arc 

Utilization 

0.02 

0.0001 

0.20 

0.50 

1.00 

1.00 

Average  Arc 

Utilization 

0.32 

0.99 

0.99 

0.99 

1.00 

1.00 

Table  12.  Arc  utilization  and  performance  comparison  of  “optimal”  Abilene  designs  and  the 

real  Abilene  design. 


3.  Proportional  Changes  in  Demand 

We  demonstrate  that  with  an  “optimal”  design,  proportional  changes  in  the 
demand  matrix  lead  to  proportional  provisioning  of  additional  arc  capacity. 

Assume  the  customer  population  of  each  of  the  eleven  Abilene  POPs  increases  by 
10%.  Using  these  increased  customer  populations  as  input  to  the  gravity  model  discussed 
in  Chapter  II,  we  generate  a  new  demand  matrix  in  Table  13. 

Note  a  proportional  increase  in  customer  population  of  10%  leads  to  an  increase 
in  demand  matrix  values  of  exactly  21%. 

bst  =  ysp(s)  ytp(t)  ->  1.1  ysp(s)  1.1  ytp(t)  =  1.21  ysp(s)  ytp(t )  =  1.21  bst 


44 


ATL 

CHI 

DEN 

HOU 

IND 

KSC 

LAX 

NYC 

SUN 

SEA 

WDC 

ATL 

-1.24 

0.18 

0.04 

0.13 

0.05 

0.03 

0.23 

0.51 

0.01 

0.04 

0.04 

GHI 

0.18 

-6.35 

0.21 

0.77 

0.28 

0.18 

1.33 

2.95 

0.04 

0.21 

0.21 

DEN 

0.04 

0.21 

-1.48 

0.16 

0.06 

0.04 

0.28 

0.61 

0.01 

0.04 

0.04 

HOU 

0.13 

0.77 

0.16 

-5.00 

0.21 

0.13 

1.01 

2.24 

0.03 

0.16 

0.16 

IND 

0.05 

0.28 

0.06 

0.21 

-1.96 

0.05 

0.37 

0.81 

0.01 

0.06 

0.06 

KSC 

0.03 

0.18 

0.04 

0.13 

0.05 

-1.24 

0.23 

0.51 

0.01 

0.04 

0.04 

LAX 

0.23 

1.33 

0.28 

1.01 

0.37 

0.23 

-7.91 

3.86 

0.05 

0.28 

0.28 

NYC 

0.51 

2.95 

0.61 

2.24 

0.81 

0.51 

3.86 

-12.81 

0.10 

0.61 

0.61 

SUN 

0.01 

0.04 

0.01 

0.03 

0.01 

0.01 

0.05 

0.10 

-0.25 

0.01 

0.01 

SEA 

0.04 

0.21 

0.04 

0.16 

0.06 

0.04 

0.28 

0.61 

0.01 

-1.48 

0.04 

WDC 

0.04 

0.21 

0.04 

0.16 

0.06 

0.04 

0.28 

0.61 

0.01 

0.04 

-1.48 

Legend: 

B(s,t) 


B(s,t)  =  Demand  for  T ratTic  Gbps 


Table  13.  O-D  demand  matrix  generated  by  increasing  the  customer  population  of  all  eleven 
Abilene  POPs  by  10%,  these  values  represent  an  increase  in  bst  of  21%  from  the 

values  in  Table  6. 

We  start  with  the  “optimal”  Abilene  design  Figure  16.  We  solve  model  NP  using 
a  B  =  20,000  and  the  increased  O-D  demand  matrix  in  Table  13.  We  note  where 
additional  arc  capacity  is  provisioned.  The  solution  to  NP  appears  in  Figure  18. 
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SEA 


Figure  18.  The  solution  to  NP  with  a  proportionally  increased  O-D  demand  matrix. 

We  note  that  the  capacity  provisioned  on  most  arcs  increases.  A  comparison  of 
the  additional  capacity  provisioned  on  each  arc  appears  is  in  Table  14. 


ATL  CHI  DEN  HOU  IND  KSC  LAX  NYC  SUN  SEA  WDC  ATL  CHI  DEN  H0U  IND  KSC  ^  NYC  SUN  SEA  WDC 

ATL 

CHI 

DEN 

HOU 

IND 

KSC 

LAX 

NYC 

SUN 

SEA 

WDC 


18.56 

0.98 

0.98 

2.14 

2.14 

0.88 

1.65 

7.90 

7.90 

1.68 

1.68 

0.85 

0.85 

1.88 

1.88 

4.78 

4.78 

0.40 

0.40 

2.14 

2.14 

2.08 

2.08 

1.68 

1.68 

0.68 

1.00 

8.16 

8.16 

1.46 

1.46 

6.70 

6.70 

5.98 

5.98 

8.16 

8.16 

2.27 

2.27 

1.46 

1.46 

1.24 

1.24 

0.85 

0.85 

1.24 

1.24 

0.98 

0.98 

2.27 

2.27 

0.28 

0.28 

0.84 

0.84 

1.75 

1.75 

1.65 

1.65 

6.09 

6.09 

1.36 

1.36 

0.66 

0.66 

1.19 

1.19 

3.78 

3.78 

0.31 

0.31 

1.72 

1.72 

1.87 

1.87 

1.33 

1.33 

1.00 

1.00 

6.74 

6.74 

1.23 

1.23 

5.90 

5.90 

3.97 

3.97 

6.77 

6.77 

1.88 

1.88 

1.20 

1.20 

1.05 

1.05 

0.69 

0.69 

1.02 

1.02 

0.81 

0.81 

1.91 

1.91 

Legend: 

X(s,t)  X(s,t)  =  Arc  Flow,  X(origin, destination)  Gbps 
U(s,t)  U(s,t)  =  Arc  Capacity,  U(origin, destination)  Gbps 
Utilization  Scale 


Legend: 

X(s,t)  X(s,t)  =  Arc  Flow,  X(origin, destination)  Gbps 
U(s,t)  U(s,t)  =  Arc  Capacity,  U(origin, destination)  Gbps 


Utilization  Scale 


0.0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1.0 


0.0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1.0 


Table  14.  Capacity  matrix  for  original  O-D  demand  matrix  (left),  capacity  matrix  for 

increased  O-D  demand  matrix  (right). 
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By  comparing  the  two  arc  capacity  matrices,  we  can  see  that  a  proportional 
increase  in  demand  leads  to  a  near  proportional  distribution  of  arc  capacity  over  the 
existing  arcs.  No  new  arcs  are  created;  however,  capacity  is  increased  on  existing  arcs  to 
meet  new  demand. 
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IV.  NUMERICAL  EXPERIMENTS 


A.  MULTI-PERIOD  CHANGES  IN  DEMAND 

We  consider  the  evolution  of  an  ISP’s  network  over  multiple  time  periods.  We 
compare  three  distinct  network  design  techniques  utilizing  the  network  models  discussed 
in  Chapter  II.  The  three  design  techniques  include  a  heuristic  technique,  a  myopic 
technique,  and  an  omniscient  technique. 

1.  Baseline  Abilene  Network 

We  use  the  existing  Abilene  Network  discussed  in  Chapter  III  as  the  baseline  for 
each  network  design  experiment.  This  network  has  28  arcs,  each  provisioned  with  10 
Gbps  of  capacity,  that  connect  eleven  cities  throughout  the  United  States.  An  illustration 
of  this  baseline  network  is  in  Chapter  III,  Figure  9. 

2.  Demand  Matrices 

We  consider  a  horizon  of  five  years,  each  divided  into  quarters,  for  a  total  of  T  = 
20  time  periods.  Let  BT  denote  the  O-D  demand  matrix  in  period  r  =  1,2,  ...  ,20.  We 
generate  20  synthetic  demand  matrices  (B  ,  B  ,  ...  ,  B  ),  one  for  each  period  in  the 
experiment.  Specifically,  we  model  incremental  changes  in  the  customer  population  of 
each  POP  from  one  period  to  the  next  and  generate  the  next  quarter’s  O-D  demand  matrix 
using  the  updated  customer  populations  as  input  to  the  gravity  model  discussed  in 
Chapter  II.  A  side-by-side  comparison  of  the  assumed  customer  population  of  Abilene 
POPs  at  t  =  1  and  r  =  20  appears  in  Table  15. 
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Point  of  Presence 

Population 

(millions) 

Assumed 

Customer 

Population 

(millions) 

T  =1 

Assumed 

Customer 

Population 

(millions) 

t  —  20 

Atlanta  (ATL) 

0.50 

0.16 

0.25 

Chicago  (CHI) 

2.90 

0.93 

1.46 

Denver  (DEN) 

0.60 

0.19 

0.31 

Houston  (HOU) 

2.20 

0.7 

1.19 

Indianapolis  (IND) 

0.80 

0.26 

0.43 

Kansas  City  (KSC) 

0.50 

0.16 

0.25 

Los  Angeles  (LAX) 

3.80 

1.22 

1.81 

New  York  (NYC) 

8.40 

2.69 

4.31 

Sunnyvale  (SUN) 

0.10 

0.03 

0.05 

Seattle  (SEA) 

0.60 

0.19 

0.29 

Washington  D.  C.  (WDC) 

0.60 

0.19 

0.34 

Table  15.  Assumed  customer  population  of  Abilene  POPs  at  z  =1  and  z  =  20;  population 

data  from  U.S.  Census  Bureau  (2010). 

3.  Budget  Resources  on  Costs 

We  assume  a  yearly  budget  of  25%  of  the  replacement  cost  for  the  baseline 
infrastructure  of  Abilene.  We  infer  the  cost  to  build  and  provision  each  arc  using  the  cost 
equation  from  Chapter  III.  We  divide  the  yearly  budget  into  a  per  period  budget  ( PPB ). 
We  endow  the  ISP  with  a  PPB  of  20,000. 
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4.  Heuristic  Technique 

We  first  consider  a  heuristic  design  technique  that  is  purely  reactive.  In  each  time 
period,  the  ISP  solves  model  TE  to  obtain  the  minimum  cost  routes,  and  then  it  uses  its 
available  budget  to  increase  the  capacity  of  arcs  that  are  saturated  by  the  min-cost  flows. 
The  rationale  for  this  technique  is  that  increasing  the  capacity  of  an  arc  that  is  not 
saturated  will  not  reduce  ZTe-  However,  increasing  the  capacity  of  an  arc  that  was 
saturated  by  the  min-cost  flows  could  reduce  ZTe- 

The  heuristic  rule  for  allocating  budget  to  provision  arcs  is  as  follows.  Using  the 
current  quarter’s  O-D  demand  matrix  as  input,  we  solve  model  TE  and  observe  the 
resulting  network  traffic.  The  heuristic  then  identifies  any  arcs  that  are  saturated,  and  it 
invests  the  PPB  in  an  equal  amount  of  additional  capacity  to  provision  on  the  saturated 
arcs.  For  example,  if  there  are  4  saturated  arcs,  the  heuristic  invests  the  PPB  to  increase 
the  capacity  on  these  arcs  by  an  equal  amount.  If  there  are  no  saturated  arcs,  the  heuristic 
does  not  invest  the  PPB.  After  provisioning  additional  capacity,  the  heuristic  resolves 
model  TE  for  new  flows  at  ZTe-  The  heuristic  technique  assumes  saturated  arcs  need 
additional  capacity  for  future  quarters. 

5.  Myopic  Technique 

We  next  consider  a  myopic  network  design  technique  in  which  the  ISP  invests  its 
available  budget  optimally  for  the  current  demand.  However,  the  ISP  has  no  knowledge 
of  future  demands.  Given  the  current  O-D  demand  matrix,  we  solve  model  NP  to 
optimally  provision  existing  arcs  and/or  build  new  arcs  in  the  network  in  order  to  route 
traffic  from  its  origin  to  its  destination  in  the  least  expensive  manner.  Thus,  the  ISP 
designs  the  network  optimally  for  the  current  demand,  but  it  cannot  anticipate  future 
demand. 


6.  Omniscient  Technique 

Finally,  we  consider  a  network  design  technique  that  is  omniscient,  in  the  sense 
that  we  assume  the  ISP  has  knowledge  of  current  demand  and  all  future  demands. 
Moreover,  we  assume  that  the  ISP  can  invest  its  budget  for  all  T  =  20  periods  upfront. 
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Specifically,  given  demand  matrices  for  the  current  period  and  all  future  periods,  we 
solve  model  MPNP  to  optimally  provision  network  arcs  to  minimize  the  aggregate 
routing  cost  over  all  time  periods.  The  omniscient  design  provides  a  lower  bound  on  cost 
minimization,  since  it  represents  the  best  an  ISP  can  do. 

7.  Performance  Evaluation  of  Three  Different  Network  “Design” 
Techniques  Over  20  Periods 

Our  design  techniques  result  in  three  different  network  topologies  at  x  =  20.  They 
appear  in  Figure  19.  Table  16  shows  the  arc  capacities  and  min-cost  flows  for  each 
topology. 
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Figure  19.  Heuristic  design  (top),  myopic  design  (center),  omniscient  design  (bottom), 
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topologies  at  z  =  20 


ATL  CHI  DEN  HOU  IND  KSC  LAX  NYC  SUN  SEA  WDC 


ATL 

CHI 

DEN 

HOU 

IND 

KSC 

lax 

NYC 

SUN 

SEA 

WDC 


15.20 

42.87 

10.00 

15.53 

16.02 

12.45 

80.38 

12.56 

26.17 

5.23 

10.00 

10.00 

2.51 

10.00 

15.20 

42.87 

7.64 

10.00 

14.95 

16.02 

10.00 

12.45 

80.38 

11.28 

61.01 

5.30 

10.00 

7.57 

10.00 

11.28 

61.01 

15.02 

16.02 

10.00 

12.56 

26.17 

15.93 

65.40 

10.00 

1  34 

10.00 

10.00 

2.44 

10.00 

10.00 

15.53 

16.02 

15.93 

65.40 

X(s.t) 

U(S.t) 


X(s,t)  =  Arc  Flow,  X(origin, destination)  Gbps 
U(s,t)  =  Arc  Capacity,  U(origin, destination)  Gbps 


Utilization  Scale 


ATL 

CHI 

DEN 

HOU 

IND 

KSC 

LAX 

NYC 

SUN 

SEA 

WDC 


ATL 

CHI 

DEN 

HOU 

IND 

KSC 

LAX 

NYC 

SUN 

SEA 

WDC 

14.74 

0.06 

6.17 

160.91 

14.74 

10.00 

0.11 

8.90 

10.00 

0.36 

4.09 

4.37 

6.30 

1.76 

0.36 

4.94 

4.37 

10.00 

3.88 

11.52 

1.76 

4  45 

0.78 

1.22 

10.76 

1.79 

2.16 

9.52 

0.78 

1.22 

10.76 

3.34 

10.00 

10.00 

5.85 

2.14 

0.45 

0.51 

14.04 

10.00 

3.14 

0.45 

1.27 

10.00 

14.04 

1.06 

0.51 

1.86 

015 

10.00 

10.00 

1.06 

1.09 

10.00 

1.86 

0.96 

0.06 

11.68 

0.19 

3.97 

10.00 

10.00 

10.00 

11.68 

0.93 

12.38 

3.04 

0.88 

12.38 

10.00 

10.00 

0.88 

13.95 

9.12 

1.86 

1.08 

1.48 

13.95 

10.00 

1.86 

1.85 

10.00 

10.00 

10.00 

10.00 

2.51 

0.53 

10.00 

0.92 

10.00 

111 

0.73 

015 

0.09 

1.48 

10.00 

0.78 

0.99 

0.14 

10.00 

Legend: 


X(s,t) 

U(s,t) 


X(s,t)  =  Arc  Flow,  X(origin, destination)  Gbps 
U(s,t)  =  Arc  Capacity,  U(origin, destination)  Gbps 


0.0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1.0 


Utilization  Scale 


0.0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1.0 


ATL 

CHI 

DEN 

HOU 

IND 

KSC 

LAX 

NYC 

SUN 

SEA 

WDC 


ATL 

CHI 

DEN 

HOU 

IND 

KSC 

LAX 

NYC 

SUN 

SEA 

WDC 

2.54 

1.09 

1.77 

10.00 

10.00 

1.09 

10.00 

1.88 

1.97 

2.49 

6.83 

0.50 

1.88 

10.00 

2.49 

10.00 

0.50 

1.49 

1.63 

10.00 

1.63 

10.00 

10.00 

2.65 

1.74 

0.46 

1  35 

4  47 

4.75 

10.00 

1.74 

0.46 

10.00 

10.00 

4.75 

1.98 

0.46 

1.40 

0.71 

1.77 

10.00 

10.00 

0.46 

10.00 

0.71 

1.77 

1.41 

1.54 

1  29 

1.08 

10.00 

10.00 

10.00 

1.08 

2.63 

4  25 

0.71 

7.18 

1  69 

0.52 

2.63 

10.00 

0.71 

7.18 

10.00 

0.52 

1.09 

6.82 

1.63 

4.75 

1.77 

1.08 

7.18 

1.84 

2.63 

1.09 

10.00 

1.63 

4.75 

1.77 

1.08 

7.18 

1.84 

10.00 

1.61 

1  00 

10.00 

10.00 

10.00 

1.84 

10.00 

1.84 

10.00 

1  78 

0.50 

0.52 

2.62 

10.00 

0.50 

0.52 

10.00 

Legend: 

X(s,t)  X(s,t)  =  Arc  Flow,  X(origin, destination)  Gbps 
U(s,t)  U(s,t)  =  Arc  Capacity,  U(origin, destination)  Gbps 

Utilization  Scale 


0.0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1.0 


Table  16.  Arc  capacities  and  network  flows  at  r  =  20  for  heuristic  design  (top  left),  myopic 

design  (top  right),  and  omniscient  design  (bottom  center). 


The  heuristic  design  technique  maintains  the  original  topology  of  the  baseline 


Abilene  network.  The  myopic  design  technique  builds  32  additional  arcs  to  the  baseline 
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Abilene  network.  The  omniscient  design  technique  builds  26  additional  arcs.  The 
omniscient  design  performs  at  nearly  half  the  cost  of  the  myopic  design  technique  and 
nearly  a  third  of  the  cost  of  the  heuristic  design  techniques;  however,  the  arc  utilization 
on  each  network  is  comparable.  A  comparison  of  arc  utilization  is  in  Table  17.  The 
cumulative  performance  of  the  three  design  techniques  appears  in  Figure  20. 


Heuristic 

Design 

Myopic 

Design 

Omniscient 

Design 

Number  of 

Arcs 

28 

60 

54 

r  =  20 

Maximum 

Arc 

Utilization 

0.97 

1.00 

1.00 

r  =20 

Minimum  Arc 

Utilization 

0.05 

0.00 

0.06 

t  =  20 

Average  Arc 

Utilization 

0.37 

0.47 

0.59 

Cumulative 

Performance 

(Z)  at  t  =  20 

3,064 

2,091 

1,326 

Table  17.  Arc  utilization  of  min-cost  flows  at  z  =  20  on  networks  resulting  from  each  of  the 

three  network  design  techniques. 
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Cumulative  Performance  (Z)  of  Three  Network 
"Design"  Techniques  PPB  =  20,000,  a  =  10 


9  10  11  12  13  14  15  16  17  18  19  20 


Figure  20.  Cumulative  performance  of  three  network  design  techniques.  For  example, 
the  value  at  r  =  10  on  the  heuristic  line  =  1326  is  the  cumulative  performance 
(minimum  cost)  of  the  heuristic  design  technique  over  10  time  periods.  The  value 
at  r  =  10  on  the  traffic  demand  in  Gbps  line  =  55  is  the  total  demand  for  traffic  at 

time  period  10. 

In  this  experiment,  the  three  design  techniques  result  in  different  network 
topologies  and  therefore  different  performance  values.  The  omniscient  design 
significantly  outperforms  the  heuristic  and  myopic  designs.  The  omniscient  design 
benefits  from  having  knowledge  of  all  demand  matrices  in  all  periods  and  having  access 
to  the  aggregate  budget  for  all  periods.  Thus,  it  is  able  to  build  and  provision  arcs 
optimally  over  all  periods.  The  myopic  technique  can  build  arcs  and  provision  them 
optimally;  however,  it  knows  only  the  current  demand  matrix  and  can  spend  only  a  single 
PPB.  Thus,  it  is  at  a  disadvantage  relative  to  the  omniscient  design  technique.  The 
heuristic  design  technique  only  adds  capacity  to  existing  arcs  to  improve  performance 
and  is  unable  to  benefit  from  building  new  arcs.  These  results  highlight  the  importance 
of  building  new  network  infrastructure  and/or  gaining  knowledge  of  future  customer 
demands. 
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B.  SENSITIVITY  ANALYSIS 

We  consider  the  sensitivity  of  the  previous  experiment’s  results  to  changes  in  the 
per  period  budget  (PPB)  and  the  fixed  cost  of  building  new  arcs  (a).  Either  increasing 
the  PPB  or  decreasing  a  will  allow  the  ISP  to  build  more  arcs.  We  consider  how  these 
changes  effect  performance  and  network  topology. 

1.  Varying  PPB 

First,  we  rerun  the  experiment  using  three  PPB  values:  8,000,  20,000,  and  45,000. 
The  corresponding  performance  of  the  three  design  techniques  appear  in  Figure  21. 
Figure  22  shows  a  comparison  of  the  resulting  topologies. 

We  observe  that  the  performance  of  the  heuristic  is  unaffected  by  an  increase  in 
PPB.  The  cumulative  cost  of  network  flows  over  all  T  =  20  time  periods  is  ZHeur  = 
3,064,  and  this  does  not  change  with  increased  budget.  This  result  indicates  that  the 
objective  value  ZTe  achieved  with  PPB  =  8,000  cannot  be  improved  without  building 
additional  arcs.  The  ISP  will  only  waste  these  additional  resources  unless  it  builds  new 
arcs.  In  contrast,  the  performance  of  the  myopic  and  omniscient  techniques  improves  as 
PPB  increases.  Table  18  summarizes  these  costs.  The  performance  of  the  myopic 
technique  approaches  that  of  the  heuristic  technique  as  PPB  decreases,  and  it  approaches 
that  of  the  omniscient  technique  as  PPB  increases.  This  result  highlights  the  intuitive 
effect  budget  has  on  the  ability  of  an  ISP  to  improve  performance.  If  an  ISP  has  more 
resources,  shown  here  by  a  larger  PPB,  then  the  ISP  is  better  able  to  improve 
performance.  If  an  ISP  has  fewer  resources,  shown  here  by  a  smaller  PPB,  then  the  ISP 
is  less  able  to  improve  performance.  This  result  also  highlights  the  value  of  information. 
If  the  ISP  has  knowledge  of  future  demand,  such  as  in  the  omniscient  technique,  then  it 
can  improve  performance  by  investing  where  demands  increase. 
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Table  18. 


PPB  = 
8,000 

PPB  = 
20,000 

PPB  = 
45,000 

Heuristic 

3,064 

3,064 

3,064 

Myopic 

2,817 

2,091 

1,395 

Omniscient 

1,693 

1,326 

1,162 

Cumulative  cost  of  min-cost  network  flows  for  all  T  =  20  time  periods  under  three 
design  techniques  and  with  different  budgets. 
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Figure  2 1 .  Cumulative  performance  of  network  design  technique  sensitivity  to  changes  in 

PPB :  PPB  =  8,000  (top),  PPB  =  20,000  (center),  PPB  =  45,000  (bottom).  In  all 
three  cases,  the  performance  of  the  heuristic  design  technique  is  the  same.  With 
increased  budget,  the  myopic  design  performs  closer  to  the  omniscient  design. 
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Figure  22.  Comparison  of  resulting  topologies  at  r  =  20  with  PPB  =  8,000,  20,000,  and  45,000.  With  increased  budget,  both 

the  myopic  and  omniscient  designs  yield  topologies  that  are  more  “mesh-like.” 
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Note  in  Figure  22  that  the  utilization  of  the  arcs  on  the  heuristic  design  decreases 
with  increased  PPB,  while  the  ZTe  from  Figure  21  remains  constant.  This  implies  that 
spending  resources  over  PPB  =  8,000  will  only  decrease  the  utilization  of  arcs  on  the 
network  but  will  not  improve  performance.  The  ISP  would  be  better  off  not  spending 
additional  resources  in  arc  capacity.  Note  in  Figure  22  that  the  myopic  and  omniscient 
design  improve  their  performance  by  building  additional  arcs.  As  the  PPB  increases,  the 
resulting  topology  looks  more  like  a  mesh  design. 

2.  Varying  a 

We  rerun  the  experiment  with  different  fixed  costs  for  building  new  arcs, 
specifically  a  =  20,  10,  and  5.  The  corresponding  performance  of  the  three  design 
techniques  appear  in  Figure  23.  Figure  24  shows  a  comparison  of  the  resulting 
topologies. 

Note,  that  the  performance  of  the  heuristic  is  unaffected  by  an  increase  in  a.  This 
result  is  intuitive  since  the  heuristic  design  technique  does  not  allow  the  ISP  to  build 
additional  arcs.  The  myopic  and  omniscient  techniques  both  improve  performance  as  a 
decrease.  Table  19  summarizes  these  costs.  The  performance  of  the  myopic  technique 
approaches  heuristic  performance  as  a  increases  and  approaches  omniscient  performance 
as  a  decreases.  This  result  highlights  the  effect  costs  have  on  the  ability  of  an  ISP  to 
improve  performance  because  they  can  build  more  arcs  with  the  same  budget.  If  costs 
decrease,  shown  here  by  a  lower  a,  then  ISPs  are  better  able  to  improve  performance;  if 
costs  increase,  shown  here  by  a  higher  a,  then  ISPs  are  less  able  to  improve  performance. 
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Table  19. 


a  =  20 

a  =  10 

a  =  5 

Heuristic 

3,064 

3,064 

3,064 

Myopic 

2,801 

2,091 

1,568 

Omniscient 

1,495 

1,326 

1,178 

Cumulative  cost  of  min-cost  flows  for  all  T  =  20  time  periods  under  three 
different  design  techniques  and  with  different  fixed  costs  for  new  arc 

construction. 
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Figure  23.  Cumulative  performance  of  network  design  technique  sensitivity  to  changes  in 
a:  a  =  20  (top),  a  =  10  (center),  a  =  5  (bottom).  In  all  three  cases,  the 
performance  of  the  heuristic  design  technique  is  the  same.  With  decreased  costs, 
the  myopic  design  performs  more  like  the  omniscient  design. 
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Figure  24.  Comparison  of  resulting  topologies  at  r  =  20  with  a  =  20,  10,  and  5.  With  decreased  costs,  both  the  myopic  and 

omniscient  design  yield  topologies  that  are  more  “mesh-like.” 


64 


Note  in  Figure  24  the  utilization  of  the  arcs  on  the  network  that  results  from 
heuristic  design  remains  constant;  this  is  intuitive  since  the  PPB  is  fixed.  Note  in  Figure 
24  that  as  a  decreases  the  myopic  and  omniscient  techniques  improve  performance  by 
building  and  provisioning  additional  arcs  with  the  available  PPB.  As  a  decreases,  the 
resulting  topology  looks  more  like  a  mesh  design. 

This  sensitivity  analysis  has  demonstrated  the  effects  of  budget  and  costs  on  ISP 
performance  and  topology.  Intuitively,  an  increase  in  budget  allows  an  ISP  to  improve 
performance;  likewise,  a  decrease  in  costs  allows  an  ISP  to  improve  performance.  As  the 
budget  of  an  ISP  increases  and/or  costs  decrease,  to  improve  performance,  the  topology 
of  an  ISP  approaches  a  mesh  design. 

C.  INVESTMENT  HORIZONS 

We  next  consider  the  following  question:  When  should  the  ISP  invest  in  new 
infrastructure,  and  how  much  should  it  invest?  We  define  an  investment  horizon  to  be 
the  interval  between  successive  investments  in  network  provisioning.  Keeping  the  total 
investment  the  same,  we  consider  the  investment  horizons  in  Table  20. 
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Length  of  Investment  Horizon 


1 

2 

4 

5 

10 

20 

T  =  1 

PPB 

2  x  PPB 

4  x  PPB 

5  x  PPB 

10  x  PPB 

20  x  PPB 

T  =  2 

PPB 

0 

0 

0 

0 

0 

T  =  3 

PPB 

2  x  PPB 

0 

0 

0 

0 

T  =  4 

PPB 

0 

0 

0 

0 

0 

T  =  5 

PPB 

2  x  PPB 

4  x  PPB 

0 

0 

0 

T  =  6 

PPB 

0 

0 

5  x  PPB 

0 

0 

T  =  7 

PPB 

2  x  PPB 

0 

0 

0 

0 

T  =  8 

PPB 

0 

0 

0 

0 

0 

T  =  9 

PPB 

2  x  PPB 

4  x  PPB 

0 

0 

0 

T=  10 

PPB 

0 

0 

0 

0 

0 

T=  11 

PPB 

2  x  PPB 

0 

5  x  PPB 

10  x  PPB 

0 

<N 

II 

PPB 

0 

0 

0 

0 

0 

II 

PPB 

2  x  PPB 

4  x  PPB 

0 

0 

0 

> 

II 

PPB 

0 

0 

0 

0 

0 

*0 

II 

PPB 

2  x  PPB 

0 

0 

0 

0 

II 

PPB 

0 

0 

5  x  PPB 

0 

0 

T=  17 

PPB 

2  x  PPB 

4  x  PPB 

0 

0 

0 

T  =  18 

PPB 

0 

0 

0 

0 

0 

T  =  19 

PPB 

2  x  PPB 

0 

0 

0 

0 

<N 

II 

PPB 

0 

0 

0 

0 

0 

Table  20.  Amount  invested  in  period  r  given  a  specific  investment  horizon  length 

1.  Varying  Investment  Horizons  with  the  Omniscient  Technique 


In  previous  experiments,  the  omniscient  design  technique  benefits  from  having  an 
investment  horizon  equal  to  20  periods.  In  this  experiment,  we  restrict  this  constraint. 
Specifically,  we  ask  how  the  omniscient  design  technique  will  perform  with  reduced 
investment  horizons.  We  rerun  the  experiment  for  the  omniscient  design  technique  with 
the  investment  horizons  in  Table  20  and  analyze  the  changes  in  performance  and  network 
topology. 


We  plot  the  performance  of  the  omniscient  design  technique  with  varying 
investment  horizons  in  Figure  25.  Note  the  performance  of  the  omniscient  design 
technique  with  a  20-period  investment  horizon  is  equivalent  to  the  performance  of  the 
omniscient  design  technique  from  previous  experiments.  The  cumulative  performance 
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versus  period  z  appears  in  Figure  26.  The  cumulative  performance  versus  varying 
investment  horizons  appears  in  Figure  27.  Figure  28  shows  a  comparison  of  the  resulting 
topologies  at  r  =  20. 

We  observe  in  Figure  25  and  in  Figure  27  that  reducing  the  investment  horizon 
deteriorates  performance.  This  is  further  evident  in  Figure  26  by  plotting  cumulative 
performance. 

The  results  show  that  if  an  ISP  has  knowledge  of  future  customer  demands,  as  it 
does  in  the  omniscient  design  technique,  then  the  ISP  can  improve  performance  by 
investing  resources  as  soon  as  possible  in  augmenting  arc  capacity  and/or  building  new 
arcs.  Any  delay  in  investing  resources,  demonstrated  here  by  decreasing  the  investment 
horizons,  can  only  deteriorate  performance. 


Figure  25.  Performance  of  omniscient  network  design  with  varying  investment  horizons 

versus  period  z. 
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Cumulative  Performance  (Z)  of  Omniscient  Network 
"Design"  versus  Period  r 


Figure  26.  Cumulative  performance  of  omniscient  network  design  with  varying 

investment  horizons  versus  period  r. 


Cumulative  Performance  (Z)  of  Omniscient  Network  "Design" 
at  r  =  20  versus  varying  Investment  Horizons 


Investment  Horizon 


Figure  27.  Cumulative  performance  of  omniscient  network  design  versus  varying 

investment  horizons. 
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Figure  28.  Omniscient  network  design  topologies  at  r  =  20  from  varying  investment 
horizons.  The  longer  the  investment  horizon,  the  better  the  ISP  is  able  to  invest 

optimally  for  future  demand. 
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We  observe  in  Figure  28  that  as  investment  horizons  increase  from  a  single-period 
to  20-periods,  the  ISP  builds  longer  arcs.  The  single-period  topology  has  many  short 
saturated  arcs.  As  investment  horizons  increase,  the  ISP  has  access  to  more  resources 
and  builds  longer  arcs.  This  result  highlights  the  value  of  having  access  to  resources  with 
knowledge  of  future  customer  demands.  If  an  ISP  has  knowledge  of  future  customer 
demands,  as  it  does  in  the  omniscient  design  technique,  then  it  can  improve  performance 
by  provisioning  and/or  building  arcs  to  satisfy  optimally  current  and  future  demands. 

2.  Varying  Investment  Horizons  with  the  Myopic  Technique 

In  previous  experiments,  the  myopic  design  technique  can  spend  its  PPB  one 
period  at  a  time.  In  this  experiment,  we  relax  that  constraint.  Specifically,  we  ask 
whether  the  ISP  can  do  better  by  spending  larger  amounts,  but  less  frequently.  We  rerun 
the  experiment  for  the  myopic  design  technique  with  the  investment  horizons  specified  in 
Table  20  and  examine  the  effects  on  performance  and  topology. 

We  plot  the  performance  of  the  myopic  design  technique  with  varying  investment 
horizons  in  Figure  29.  Note  the  performance  of  the  myopic  design  technique  with  a 
single-period  investment  horizon  is  equivalent  to  the  performance  of  the  myopic  design 
technique  from  previous  experiments.  The  cumulative  performance  for  each  time  period 
appears  in  Figure  30.  The  cumulative  performance  for  each  investment  horizon  appears 
in  Figure  31.  Figure  32  shows  a  comparison  of  the  resulting  topologies  at  x  =  20. 

We  observe  in  Figure  31  that  increasing  the  investment  horizon  initially  improves 
performance  but  then  it  deteriorates  as  investment  horizons  become  too  long.  The 
performance  continuously  improves  as  the  investment  horizon  increases  from  1 -period  to 
4-periods;  however,  when  the  investment  horizon  reaches  5-periods,  performance  begins 
to  deteriorate.  From  this  point  on,  increasing  the  investment  horizon  will  result  in  further 
deteriorating  performance. 

Increasing  the  investment  horizon  can  benefit  the  ISP  in  two  ways.  First,  because 
a  longer  period  of  time  has  elapsed  between  investments,  the  ISP  gains  knowledge  of 
customer  demands  before  investing.  Second,  the  ISP  has  a  larger  quantity  of  resources  to 
invest  in  meeting  customer  demands.  For  example,  with  a  4-period  investment  horizon, 
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the  ISP  invests  4  PPB  in  period  1  and  then  becomes  aware  of  customer  demands  in  time 
periods  2,  3,  and  4  before  investing  again.  This  delay  in  investment  allows  the  ISP  to 
gain  knowledge  of  where  customer  demands  have  increased.  The  ISP  also  has  more  PPB 
in  time  period  5.  With  an  investment  horizon  of  4-periods,  the  ISP  has  access  to  4  PPB. 

There  is  a  limit  to  the  benefits  of  increasing  an  ISP’s  investment  horizon.  For 
example,  with  a  20-period  investment  horizon,  although  the  ISP  has  access  to  20  PPB  in 
period  1,  it  only  has  knowledge  of  customer  demands  in  period  1  and  can  only  investment 
optimal  for  that  period.  After  period  1 ,  performance  deteriorates  quickly  relative  to  the 
other  investment  horizons.  This  is  evident  in  Figure  29.  The  ISP  must  find  a  balance 
between  realizing  customer  demands  and  investing  in  these  realized  demands. 

The  myopic  design  technique  with  4-period  and  5-period  investment  horizons 
performs  well  relative  to  the  omniscient  technique.  The  myopic  design  technique  with  4 
and  5-period  investment  horizons  significantly  outperforms  the  myopic  design  from 
previous  experiments.  These  results  indicate  that  without  knowledge  of  future  demand, 
as  is  the  case  in  the  myopic  technique,  an  ISP  can  improve  performance  by  choosing  a 
proper  investment  horizon.  These  results  are  important  because  ISPs  typically  do  not 
have  knowledge  of  future  customer  demands. 


71 


Figure  29. 


Figure  30. 


Performance  of  myopic  network  design  with  varying  investment  horizons 

versus  period  r. 


Cumulative  Performance  (Z)  of  Myopic  Network 


Cumulative  performance  of  myopic  network  design  with  varying  investment 
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horizons  versus  period  z. 


Figure  31. 


Cumulative  performance  of  myopic  network  design  versus  varying  investment 

horizons. 
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Figure  32.  Myopic  network  design  topologies  at  z  =  20  from  varying  investment 

horizons.  With  periodic  investment,  the  ISP  balances  uncertainty  in  demand  with 

a  larger  investment  budget. 
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In  Figure  32,  we  observe  that  there  is  no  significant  change  in  the  topology  of  a 
myopic  design  technique  with  investment  horizons. 

We  vary  the  investment  horizon  in  our  experiments  and  analyze  the  effects  on 
performance  and  topology.  For  the  omniscient  design  technique,  we  observe  that 
performance  can  only  deteriorate  by  decreasing  the  investment  horizon.  These  results 
support  the  value  of  knowing  future  customer  demands.  If  ISPs  can  gain  knowledge  of 
where  customer  demands  for  their  services  will  increase,  they  can  improve  future 
performance  by  investing  resources  as  soon  as  possible. 

For  the  myopic  design  technique,  we  observe  that  ISPs  can  improve  performance 
by  selecting  an  investment  horizon  that  allows  spending  resources  after  realizing 
customer  demands.  This  allows  the  ISP  to  achieve  performance  that  approaches 
omniscient  performance.  These  findings  are  useful  because  ISPs  typically  do  not  have 
knowledge  of  future  customer  demands;  however,  they  can  implement  policies  that 
mimic  the  myopic  technique  discussed  here. 
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V.  CONCLUSIONS  AND  FUTURE  WORK 


In  this  thesis,  we  model  ISP  decision  making  via  a  traffic  engineering  model,  a 
network  provisioning  model,  and  a  multi-period  network  provision  model.  These  models 
capture  ISP  decision  making  in  the  presence  of  changing  environmental  conditions  such 
as  customer  population  growth,  changing  infrastructure  costs,  and  ISP  budget  constraints. 

We  show  that  as  customer  demand  increases,  ISPs  must  increase  the  capacity  on 
available  arcs  or  build  new  arcs  to  handle  the  increase  in  customer  demand.  If  ISPs  fail 
to  make  these  changes,  demand  will  increase  to  the  point  where  the  current  network 
infrastructure  can  no  longer  handle  demand.  We  also  show  that  traffic  routing  is  not 
intuitive.  Different  objectives  lead  to  different  “optimal”  routes.  In  this  thesis,  we 
assume  the  ISP  wants  to  minimize  operating  cost,  and  this  drives  traffic  routing  to  prefer 
shorter  traffic  routes. 

Using  an  assumed  demand  matrix  inferred  from  customer  populations,  we 
develop  an  “optimally  designed”  Abilene  and  compare  it  to  the  real  Abilene  network. 
While  the  “optimal”  Abilene  outperforms  the  real  Abilene,  the  resulting  topology  is 
“optimal”  only  for  that  demand.  This  exercise  helps  to  focus  on  the  factors  that  influence 
performance  such  as  infrastructure  cost. 

We  consider  the  performance  and  resulting  network  topologies  of  three  network 
design  techniques  on  a  set  of  20  synthetic  demand  matrices.  We  present  a  heuristic 
design  technique  that  is  reactive  to  changing  customer  demands  and  augments  capacity 
where  arcs  reach  saturation.  We  develop  a  myopic  design  technique  that  only  has 
knowledge  of  customer  demands  and  access  to  the  ISP’s  budget  in  the  current  time 
period.  We  develop  an  omniscient  design  technique  that  has  knowledge  of  customer 
demands  and  access  to  the  ISP’s  budget  in  all  time  periods.  The  performance  and 
topologies  of  the  three  design  techniques  are  considerably  different.  The  results  show 
that  ISPs  can  improve  performance  either  by  building  network  infrastructure  or  by 
improving  knowledge  of  customer  demands. 
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We  conduct  a  sensitivity  analysis  by  varying  budget  and  costs.  Intuitively,  an 
increase  in  budget  or  a  decrease  in  costs  allows  an  ISP  to  improve  performance  by 
augmenting  capacity  and/or  building  more  arcs.  With  greater  budget,  network  topologies 
look  more  like  mesh  designs  as  ISPs  invest  to  improve  performance. 

We  vary  the  investment  horizons  in  our  experiment  and  focus  on  the  omniscient 
and  myopic  design  techniques.  For  the  omniscient  design  technique,  we  show  that 
reducing  the  investment  horizon  deteriorates  performance.  If  an  ISP  has  knowledge  of 
future  customer  demands,  then  it  is  better  off  investing  resources  as  soon  as  possible  to 
improve  performance  in  current  and  future  periods.  These  results  highlight  the  value  of 
knowing  future  customer  demands. 

For  the  myopic  design  technique,  we  show  that  increasing  the  investment  horizon 
can  improve  performance  and  provides  two  benefits  to  an  ISP.  First,  the  ISP  is  able  to 
realize  customer  demands  before  investing  in  augmenting  capacity  and/or  building  new 
arcs.  Second,  the  ISP  is  able  to  access  more  resources  to  invest  in  these  realized 
customer  demands.  Although  increasing  the  investment  horizon  yields  benefits,  the  ISP 
must  find  a  balance  between  realizing  customer  demands  and  investing  in  these  realized 
demands.  The  results  show  that  choosing  long  investment  horizons  can  diminish  the 
benefits.  ISPs  can  approach  omniscient  performance  by  choosing  proper  investment 
horizons.  These  findings  are  useful  because  ISPs  typically  do  not  know  future  customer 
demands;  however,  they  can  implement  policies  that  mimic  the  myopic  technique 
discussed  here  and  choose  an  investment  horizon  that  improves  performance. 

The  results  of  this  thesis  leave  many  opportunities  for  future  research.  The 
potential  benefits  of  finding  a  balance  between  how  much  to  spend  and  when  to  spend  are 
attractive  for  ISPs,  since  they  typically  do  not  have  knowledge  of  future  customer 
demands.  Further  research  will  need  to  focus  on  identifying  these  spending  policies. 

The  resulting  topologies  from  the  experiments  suggest  that  mesh  designs  provide 
good  performance  in  the  presence  of  changing  demand.  In  these  experiments,  how  would 
mesh  designs  perform  compared  to  other  network  topologies  such  as  ring,  bus,  star,  or 
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hierarchical  designs?  Research  focused  on  network  performance  across  various 
topologies  would  yield  interesting  results  and  could  lead  ISPs  to  focus  their  network 
design  to  a  specific  topology. 

The  experiments  in  this  thesis  used  a  single  set  of  20  synthetic  demand  matrices. 
But  these  demand  matrices  represent  only  a  single  trajectory  of  evolving  demand.  Would 
these  design  techniques  perform  equally  well  under  different  demand  growth?  This 
would  help  to  answer  the  question  how  sensitive  are  the  design  techniques  to  variability 
in  changing  demand. 

In  this  thesis,  we  minimize  the  cost  of  delivering  traffic,  but  other  ISP  objectives, 
such  as  minimizing  network  congestion,  may  also  be  important.  Performing  this  analysis 
for  different  objectives  might  show  other  topologies  or  design  techniques  to  be  better. 
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APPENDIX 


Synthetic  Demand  Matrices  used  in  Numerical  Experiments 

For  example,  the  row  origin  ATL,  destination  ATL,  B1  =  -1.03  is  the  total  demand 
from  all  other  POPs  to  ATL  in  period  1.  Likewise,  the  row  origin  ATL,  destination  CHI, 
B3  =  0.16  is  the  demand  for  traffic  from  ATL  to  CHI  in  period  3. 


Origin 

Destination 

B1 

B2 

B3 

B4 

B5 

B6 

B7 

B8 

B9 

B10 

ATL 

ATL 

-1.03 

-1.07 

-1.14 

-1.17 

-1.27 

-1.35 

-1.39 

-1.47 

-1.58 

-1.64 

ATL 

cm 

0.14 

0.15 

0.16 

0.17 

0.18 

0.19 

0.20 

0.21 

0.23 

0.24 

ATL 

DEN 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

ATL 

HOU 

0.11 

0.11 

0.12 

0.12 

0.13 

0.15 

0.15 

0.16 

0.17 

0.17 

ATL 

IND 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

ATL 

KSC 

0.02 

0.03 

0.03 

0.03 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

ATL 

LAX 

0.19 

0.19 

0.20 

0.21 

0.23 

0.24 

0.24 

0.26 

0.28 

0.29 

ATL 

NYC 

0.42 

0.45 

0.48 

0.49 

0.53 

0.57 

0.59 

0.61 

0.66 

0.69 

ATL 

SUN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

ATL 

SEA 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.04 

0.05 

ATL 

WDC 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

CHI 

ATL 

0.14 

0.15 

0.16 

0.17 

0.18 

0.19 

0.20 

0.21 

0.23 

0.24 

cm 

cm 

-5.25 

-5.60 

-6.06 

-6.29 

-6.82 

-7.11 

-7.39 

-7.79 

-8.26 

-8.68 

cm 

DEN 

0.17 

0.18 

0.19 

0.20 

0.22 

0.22 

0.23 

0.24 

0.25 

0.27 

cm 

HOU 

0.64 

0.68 

0.71 

0.75 

0.82 

0.87 

0.91 

0.97 

1.01 

1.03 

cm 

IND 

0.23 

0.25 

0.27 

0.28 

0.30 

0.31 

0.33 

0.34 

0.37 

0.38 

cm 

KSC 

0.14 

0.15 

0.16 

0.17 

0.18 

0.18 

0.19 

0.20 

0.22 

0.23 

cm 

LAX 

1.10 

1.14 

1.23 

1.27 

1.38 

1.41 

1.47 

1.57 

1.67 

1.76 

cm 

NYC 

2.44 

2.64 

2.91 

3.00 

3.26 

3.41 

3.55 

3.70 

3.94 

4.16 

cm 

SUN 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

cm 

SEA 

0.17 

0.18 

0.20 

0.21 

0.22 

0.23 

0.24 

0.25 

0.26 

0.27 

cm 

WDC 

0.17 

0.19 

0.19 

0.21 

0.23 

0.23 

0.25 

0.26 

0.28 

0.29 

DEN 

ATL 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

DEN 

cm 

0.17 

0.18 

0.19 

0.20 

0.22 

0.22 

0.23 

0.24 

0.25 

0.27 

DEN 

DEN 

-1.22 

-1.28 

-1.34 

-1.38 

-1.50 

-1.56 

-1.60 

-1.65 

-1.71 

-1.85 

DEN 

HOU 

0.13 

0.14 

0.14 

0.15 

0.16 

0.17 

0.17 

0.18 

0.18 

0.19 

DEN 

IND 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.06 

0.07 

0.07 

DEN 

KSC 

0.03 

0.03 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

DEN 

LAX 

0.23 

0.23 

0.24 

0.25 

0.27 

0.27 

0.28 

0.29 

0.30 

0.33 

DEN 

NYC 

0.50 

0.54 

0.57 

0.58 

0.64 

0.66 

0.68 

0.69 

0.72 

0.78 

DEN 

SUN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

DEN 

SEA 

0.04 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.05 

DEN 

WDC 

0.04 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.06 

HOU 

ATL 

0.11 

0.11 

0.12 

0.12 

0.13 

0.15 

0.15 

0.16 

0.17 

0.17 

HOU 

cm 

0.64 

0.68 

0.71 

0.75 

0.82 

0.87 

0.91 

0.97 

1.01 

1.03 

HOU 

DEN 

0.13 

0.14 

0.14 

0.15 

0.16 

0.17 

0.17 

0.18 

0.18 

0.19 

HOU 

HOU 

-4.14 

-4.39 

-4.58 

-4.82 

-5.21 

-5.58 

-5.87 

-6.12 

-6.38 

-6.61 

HOU 

IND 

0.18 

0.19 

0.19 

0.21 

0.22 

0.24 

0.25 

0.26 

0.27 

0.28 

HOU 

KSC 

0.11 

0.11 

0.12 

0.12 

0.13 

0.14 

0.14 

0.15 

0.16 

0.17 

HOU 

LAX 

0.84 

0.86 

0.89 

0.93 

1.01 

1.07 

1.12 

1.19 

1.23 

1.28 

HOU 

NYC 

1.85 

1.99 

2.10 

2.20 

2.39 

2.58 

2.71 

2.79 

2.92 

3.04 

HOU 

SUN 

0.02 

0.02 

0.02 

0.02 

0.03 

0.03 

0.03 

0.03 

0.03 

0.03 

HOU 

SEA 

0.13 

0.14 

0.15 

0.16 

0.16 

0.18 

0.18 

0.19 

0.19 

0.20 

HOU 

WDC 

0.13 

0.14 

0.14 

0.15 

0.17 

0.18 

0.19 

0.20 

0.21 

0.21 
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Origin 

Destination 

B1 

B2 

B3 

B4 

B5 

B6 

B7 

B8 

B9 

B10 

IND 

ATL 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

IND 

CHI 

0.23 

0.25 

0.27 

0.28 

0.30 

0.31 

0.33 

0.34 

0.37 

0.38 

IND 

DEN 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.06 

0.07 

0.07 

IND 

HOU 

0.18 

0.19 

0.19 

0.21 

0.22 

0.24 

0.25 

0.26 

0.27 

0.28 

IND 

IND 

-1.62 

-1.74 

-1.84 

-1.93 

-2.02 

-2.16 

-2.25 

-2.34 

-2.50 

-2.63 

IND 

KSC 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

IND 

LAX 

0.30 

0.32 

0.33 

0.35 

0.37 

0.38 

0.40 

0.42 

0.45 

0.48 

IND 

NYC 

0.67 

0.73 

0.79 

0.82 

0.86 

0.93 

0.97 

0.99 

1.07 

1.13 

IND 

SUN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

IND 

SEA 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.06 

0.07 

0.07 

0.07 

IND 

WDC 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.07 

0.07 

0.08 

0.08 

KSC 

ATL 

0.02 

0.03 

0.03 

0.03 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

KSC 

CHI 

0.14 

0.15 

0.16 

0.17 

0.18 

0.18 

0.19 

0.20 

0.22 

0.23 

KSC 

DEN 

0.03 

0.03 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

KSC 

HOU 

0.11 

0.11 

0.12 

0.12 

0.13 

0.14 

0.14 

0.15 

0.16 

0.17 

KSC 

IND 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

KSC 

KSC 

-1.02 

-1.07 

-1.11 

-1.16 

-1.23 

-1.26 

-1.32 

-1.40 

-1.50 

-1.61 

KSC 

LAX 

0.19 

0.19 

0.20 

0.21 

0.22 

0.22 

0.23 

0.25 

0.27 

0.29 

KSC 

NYC 

0.42 

0.44 

0.47 

0.48 

0.52 

0.53 

0.56 

0.58 

0.63 

0.68 

KSC 

SUN 

0.00 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

KSC 

SEA 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

KSC 

WDC 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

LAX 

ATL 

0.19 

0.19 

0.20 

0.21 

0.23 

0.24 

0.24 

0.26 

0.28 

0.29 

LAX 

cm 

1.10 

1.14 

1.23 

1.27 

1.38 

1.41 

1.47 

1.57 

1.67 

1.76 

LAX 

DEN 

0.23 

0.23 

0.24 

0.25 

0.27 

0.27 

0.28 

0.29 

0.30 

0.33 

LAX 

HOU 

0.84 

0.86 

0.89 

0.93 

1.01 

1.07 

1.12 

1.19 

1.23 

1.28 

LAX 

IND 

0.30 

0.32 

0.33 

0.35 

0.37 

0.38 

0.40 

0.42 

0.45 

0.48 

LAX 

KSC 

0.19 

0.19 

0.20 

0.21 

0.22 

0.22 

0.23 

0.25 

0.27 

0.29 

LAX 

LAX 

-6.54 

-6.80 

-7.23 

-7.49 

-8.12 

-8.39 

-8.74 

-9.20 

-9.76 

-10.36 

LAX 

NYC 

3.19 

3.36 

3.61 

3.71 

4.04 

4.18 

4.36 

4.54 

4.84 

5.17 

LAX 

SUN 

0.04 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.05 

LAX 

SEA 

0.23 

0.23 

0.25 

0.26 

0.28 

0.29 

0.29 

0.30 

0.32 

0.34 

LAX 

WDC 

0.23 

0.24 

0.24 

0.26 

0.28 

0.29 

0.30 

0.32 

0.35 

0.36 

NYC 

ATL 

0.42 

0.45 

0.48 

0.49 

0.53 

0.57 

0.59 

0.61 

0.66 

0.69 

NYC 

cm 

2.44 

2.64 

2.91 

3.00 

3.26 

3.41 

3.55 

3.70 

3.94 

4.16 

NYC 

DEN 

0.50 

0.54 

0.57 

0.58 

0.64 

0.66 

0.68 

0.69 

0.72 

0.78 

NYC 

HOU 

1.85 

1.99 

2.10 

2.20 

2.39 

2.58 

2.71 

2.79 

2.92 

3.04 

NYC 

IND 

0.67 

0.73 

0.79 

0.82 

0.86 

0.93 

0.97 

0.99 

1.07 

1.13 

NYC 

KSC 

0.42 

0.44 

0.47 

0.48 

0.52 

0.53 

0.56 

0.58 

0.63 

0.68 

NYC 

LAX 

3.19 

3.36 

3.61 

3.71 

4.04 

4.18 

4.36 

4.54 

4.84 

5.17 

NYC 

NYC 

-10.58 

-11.33 

-12.19 

-12.61 

-13.67 

-14.34 

-14.96 

-15.50 

-16.48 

-17.45 

NYC 

SUN 

0.08 

0.09 

0.09 

0.10 

0.10 

0.11 

0.11 

0.12 

0.12 

0.13 

NYC 

SEA 

0.50 

0.54 

0.60 

0.62 

0.65 

0.69 

0.71 

0.72 

0.75 

0.80 

NYC 

WDC 

0.50 

0.54 

0.57 

0.61 

0.66 

0.69 

0.73 

0.76 

0.82 

0.86 
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Origin 

Destination 

B1 

B2 

B3 

B4 

B5 

B6 

B7 

B8 

B9 

B10 

SUN 

ATL 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

SUN 

CHI 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

SUN 

DEN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

SUN 

HOU 

0.02 

0.02 

0.02 

0.02 

0.03 

0.03 

0.03 

0.03 

0.03 

0.03 

SUN 

IND 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

SUN 

KSC 

0.00 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

SUN 

LAX 

0.04 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.05 

SUN 

NYC 

0.08 

0.09 

0.09 

0.10 

0.10 

0.11 

0.11 

0.12 

0.12 

0.13 

SUN 

SUN 

-0.21 

-0.22 

-0.23 

-0.23 

-0.25 

-0.27 

-0.27 

-0.28 

-0.29 

-0.31 

SUN 

SEA 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

SUN 

WDC 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

SEA 

ATL 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.04 

0.05 

SEA 

CHI 

0.17 

0.18 

0.20 

0.21 

0.22 

0.23 

0.24 

0.25 

0.26 

0.27 

SEA 

DEN 

0.04 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.05 

SEA 

HOU 

0.13 

0.14 

0.15 

0.16 

0.16 

0.18 

0.18 

0.19 

0.19 

0.20 

SEA 

IND 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.06 

0.07 

0.07 

0.07 

SEA 

KSC 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

SEA 

LAX 

0.23 

0.23 

0.25 

0.26 

0.28 

0.29 

0.29 

0.30 

0.32 

0.34 

SEA 

NYC 

0.50 

0.54 

0.60 

0.62 

0.65 

0.69 

0.71 

0.72 

0.75 

0.80 

SEA 

SUN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

SEA 

SEA 

-1.22 

-1.30 

-1.40 

-1.47 

-1.55 

-1.62 

-1.67 

-1.71 

-1.78 

-1.89 

SEA 

WDC 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.05 

0.06 

WDC 

ATL 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

WDC 

CHI 

0.17 

0.19 

0.19 

0.21 

0.23 

0.23 

0.25 

0.26 

0.28 

0.29 

WDC 

DEN 

0.04 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.06 

WDC 

HOU 

0.13 

0.14 

0.14 

0.15 

0.17 

0.18 

0.19 

0.20 

0.21 

0.21 

WDC 

IND 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.07 

0.07 

0.08 

0.08 

WDC 

KSC 

0.03 

0.03 

0.03 

0.03 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

WDC 

LAX 

0.23 

0.24 

0.24 

0.26 

0.28 

0.29 

0.30 

0.32 

0.35 

0.36 

WDC 

NYC 

0.50 

0.54 

0.57 

0.61 

0.66 

0.69 

0.73 

0.76 

0.82 

0.86 

WDC 

SUN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

WDC 

SEA 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.05 

0.06 

WDC 

WDC 

-1.22 

-1.30 

-1.35 

-1.44 

-1.57 

-1.63 

-1.73 

-1.81 

-1.95 

-2.03 
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Origin 

Destination 

B11 

B12 

B13 

B14 

B15 

B16 

B17 

B18 

B19 

B20 

ATL 

ATL 

-1.72 

-1.77 

-1.82 

-1.85 

-2.00 

-2.11 

-2.22 

-2.35 

-2.47 

-2.58 

ATL 

CHI 

0.25 

0.25 

0.26 

0.26 

0.29 

0.29 

0.31 

0.33 

0.35 

0.36 

ATL 

DEN 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.07 

0.07 

0.07 

0.08 

ATL 

HOU 

0.18 

0.19 

0.20 

0.20 

0.22 

0.23 

0.25 

0.26 

0.28 

0.29 

ATL 

IND 

0.07 

0.07 

0.07 

0.08 

0.08 

0.09 

0.09 

0.10 

0.10 

0.11 

ATL 

KSC 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

ATL 

LAX 

0.31 

0.32 

0.32 

0.33 

0.34 

0.37 

0.38 

0.40 

0.42 

0.45 

ATL 

NYC 

0.72 

0.74 

0.76 

0.77 

0.83 

0.89 

0.93 

0.98 

1.02 

1.06 

ATL 

SUN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

ATL 

SEA 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.06 

0.07 

0.07 

ATL 

WDC 

0.05 

0.05 

0.06 

0.06 

0.06 

0.07 

0.07 

0.08 

0.08 

0.08 

cm 

ATL 

0.25 

0.25 

0.26 

0.26 

0.29 

0.29 

0.31 

0.33 

0.35 

0.36 

cm 

cm 

-8.95 

-9.38 

-9.80 

-10.07 

-10.75 

-11.21 

-11.80 

-12.31 

-12.90 

-13.49 

CHI 

DEN 

0.28 

0.30 

0.31 

0.33 

0.35 

0.37 

0.40 

0.42 

0.43 

0.46 

CHI 

HOU 

1.08 

1.14 

1.23 

1.26 

1.35 

1.41 

1.50 

1.57 

1.63 

1.73 

CHI 

IND 

0.40 

0.41 

0.44 

0.47 

0.50 

0.52 

0.54 

0.58 

0.60 

0.63 

CHI 

KSC 

0.25 

0.25 

0.27 

0.27 

0.29 

0.29 

0.31 

0.32 

0.35 

0.37 

CHI 

LAX 

1.84 

1.90 

1.96 

2.02 

2.09 

2.19 

2.28 

2.37 

2.50 

2.64 

CHI 

NYC 

4.22 

4.46 

4.62 

4.72 

5.09 

5.34 

5.58 

5.81 

6.07 

6.30 

CHI 

SUN 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.07 

0.07 

0.07 

CHI 

SEA 

0.28 

0.29 

0.31 

0.32 

0.34 

0.35 

0.37 

0.38 

0.41 

0.43 

CHI 

WDC 

0.31 

0.33 

0.34 

0.36 

0.39 

0.39 

0.42 

0.45 

0.47 

0.50 

DEN 

ATL 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.07 

0.07 

0.07 

0.08 

DEN 

cm 

0.28 

0.30 

0.31 

0.33 

0.35 

0.37 

0.40 

0.42 

0.43 

0.46 

DEN 

DEN 

-1.97 

-2.11 

-2.18 

-2.26 

-2.44 

-2.64 

-2.82 

-2.94 

-3.04 

-3.24 

DEN 

HOU 

0.21 

0.23 

0.24 

0.25 

0.27 

0.30 

0.32 

0.33 

0.34 

0.37 

DEN 

IND 

0.08 

0.08 

0.09 

0.09 

0.10 

0.11 

0.11 

0.12 

0.13 

0.13 

DEN 

KSC 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.07 

0.07 

0.07 

0.08 

DEN 

LAX 

0.36 

0.38 

0.39 

0.40 

0.42 

0.46 

0.48 

0.50 

0.52 

0.56 

DEN 

NYC 

0.82 

0.89 

0.91 

0.94 

1.02 

1.12 

1.18 

1.23 

1.27 

1.34 

DEN 

SUN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.02 

DEN 

SEA 

0.05 

0.06 

0.06 

0.06 

0.07 

0.07 

0.08 

0.08 

0.09 

0.09 

DEN 

WDC 

0.06 

0.07 

0.07 

0.07 

0.08 

0.08 

0.09 

0.10 

0.10 

0.11 

HOU 

ATL 

0.18 

0.19 

0.20 

0.20 

0.22 

0.23 

0.25 

0.26 

0.28 

0.29 

HOU 

cm 

1.08 

1.14 

1.23 

1.26 

1.35 

1.41 

1.50 

1.57 

1.63 

1.73 

HOU 

DEN 

0.21 

0.23 

0.24 

0.25 

0.27 

0.30 

0.32 

0.33 

0.34 

0.37 

HOU 

HOU 

-6.94 

-7.43 

-7.89 

-8.06 

-8.55 

-9.24 

-9.72 

-10.12 

-10.50 

-11.28 

HOU 

IND 

0.30 

0.32 

0.35 

0.36 

0.39 

0.41 

0.43 

0.46 

0.48 

0.51 

HOU 

KSC 

0.18 

0.19 

0.21 

0.21 

0.22 

0.23 

0.25 

0.26 

0.27 

0.30 

HOU 

LAX 

1.37 

1.45 

1.52 

1.56 

1.60 

1.75 

1.82 

1.89 

1.97 

2.14 

HOU 

NYC 

3.15 

3.40 

3.59 

3.65 

3.89 

4.27 

4.45 

4.62 

4.78 

5.11 

HOU 

SUN 

0.03 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.06 

HOU 

SEA 

0.21 

0.22 

0.24 

0.25 

0.26 

0.28 

0.30 

0.31 

0.32 

0.35 

HOU 

WDC 

0.23 

0.25 

0.27 

0.28 

0.30 

0.31 

0.34 

0.36 

0.37 

0.41 
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Origin 

Destination 

B11 

B12 

B13 

B14 

B15 

B16 

B17 

B18 

B19 

B20 

IND 

ATL 

0.07 

0.07 

0.07 

0.08 

0.08 

0.09 

0.09 

0.10 

0.10 

0.11 

IND 

CHI 

0.40 

0.41 

0.44 

0.47 

0.50 

0.52 

0.54 

0.58 

0.60 

0.63 

IND 

DEN 

0.08 

0.08 

0.09 

0.09 

0.10 

0.11 

0.11 

0.12 

0.13 

0.13 

IND 

HOU 

0.30 

0.32 

0.35 

0.36 

0.39 

0.41 

0.43 

0.46 

0.48 

0.51 

IND 

IND 

-2.77 

-2.89 

-3.07 

-3.21 

-3.43 

-3.64 

-3.78 

-4.02 

-4.18 

-4.43 

IND 

KSC 

0.07 

0.07 

0.07 

0.08 

0.08 

0.08 

0.09 

0.09 

0.10 

0.11 

IND 

LAX 

0.51 

0.53 

0.55 

0.58 

0.60 

0.64 

0.66 

0.69 

0.73 

0.78 

IND 

NYC 

1.17 

1.23 

1.30 

1.35 

1.45 

1.56 

1.60 

1.70 

1.77 

1.86 

IND 

SUN 

0.01 

0.01 

0.01 

0.02 

0.02 

0.02 

0.02 

0.02 

0.02 

0.02 

IND 

SEA 

0.08 

0.08 

0.09 

0.09 

0.10 

0.10 

0.11 

0.11 

0.12 

0.13 

IND 

WDC 

0.09 

0.09 

0.10 

0.10 

0.11 

0.11 

0.12 

0.13 

0.14 

0.15 

KSC 

ATL 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

KSC 

CHI 

0.25 

0.25 

0.27 

0.27 

0.29 

0.29 

0.31 

0.32 

0.35 

0.37 

KSC 

DEN 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.07 

0.07 

0.07 

0.08 

KSC 

HOU 

0.18 

0.19 

0.21 

0.21 

0.22 

0.23 

0.25 

0.26 

0.27 

0.30 

KSC 

IND 

0.07 

0.07 

0.07 

0.08 

0.08 

0.08 

0.09 

0.09 

0.10 

0.11 

KSC 

KSC 

-1.72 

-1.80 

-1.86 

-1.92 

-1.99 

-2.08 

-2.22 

-2.27 

-2.43 

-2.62 

KSC 

LAX 

0.31 

0.32 

0.33 

0.34 

0.34 

0.36 

0.38 

0.38 

0.42 

0.45 

KSC 

NYC 

0.72 

0.75 

0.77 

0.79 

0.83 

0.88 

0.93 

0.94 

1.01 

1.08 

KSC 

SUN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

KSC 

SEA 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.06 

0.07 

0.07 

KSC 

WDC 

0.05 

0.06 

0.06 

0.06 

0.06 

0.06 

0.07 

0.07 

0.08 

0.09 

LAX 

ATL 

0.31 

0.32 

0.32 

0.33 

0.34 

0.37 

0.38 

0.40 

0.42 

0.45 

LAX 

cm 

1.84 

1.90 

1.96 

2.02 

2.09 

2.19 

2.28 

2.37 

2.50 

2.64 

LAX 

DEN 

0.36 

0.38 

0.39 

0.40 

0.42 

0.46 

0.48 

0.50 

0.52 

0.56 

LAX 

HOU 

1.37 

1.45 

1.52 

1.56 

1.60 

1.75 

1.82 

1.89 

1.97 

2.14 

LAX 

IND 

0.51 

0.53 

0.55 

0.58 

0.60 

0.64 

0.66 

0.69 

0.73 

0.78 

LAX 

KSC 

0.31 

0.32 

0.33 

0.34 

0.34 

0.36 

0.38 

0.38 

0.42 

0.45 

LAX 

LAX 

-10.84 

-11.39 

-11.67 

-11.94 

-12.38 

-13.42 

-13.82 

-14.30 

-15.03 

-16.06 

LAX 

NYC 

5.35 

5.65 

5.72 

5.82 

6.04 

6.65 

6.77 

6.98 

7.32 

7.78 

LAX 

SUN 

0.06 

0.06 

0.06 

0.07 

0.07 

0.07 

0.08 

0.08 

0.08 

0.09 

LAX 

SEA 

0.35 

0.37 

0.39 

0.40 

0.41 

0.43 

0.45 

0.46 

0.49 

0.53 

LAX 

WDC 

0.39 

0.42 

0.42 

0.44 

0.46 

0.49 

0.51 

0.55 

0.57 

0.62 

NYC 

ATL 

0.72 

0.74 

0.76 

0.77 

0.83 

0.89 

0.93 

0.98 

1.02 

1.06 

NYC 

cm 

4.22 

4.46 

4.62 

4.72 

5.09 

5.34 

5.58 

5.81 

6.07 

6.30 

NYC 

DEN 

0.82 

0.89 

0.91 

0.94 

1.02 

1.12 

1.18 

1.23 

1.27 

1.34 

NYC 

HOU 

3.15 

3.40 

3.59 

3.65 

3.89 

4.27 

4.45 

4.62 

4.78 

5.11 

NYC 

IND 

1.17 

1.23 

1.30 

1.35 

1.45 

1.56 

1.60 

1.70 

1.77 

1.86 

NYC 

KSC 

0.72 

0.75 

0.77 

0.79 

0.83 

0.88 

0.93 

0.94 

1.01 

1.08 

NYC 

LAX 

5.35 

5.65 

5.72 

5.82 

6.04 

6.65 

6.77 

6.98 

7.32 

7.78 

NYC 

NYC 

-17.99 

-19.11 

-19.73 

-20.15 

-21.45 

-23.12 

-23.99 

-24.93 

-26.01 

-27.51 

NYC 

SUN 

0.13 

0.14 

0.15 

0.15 

0.17 

0.18 

0.19 

0.20 

0.20 

0.21 

NYC 

SEA 

0.81 

0.86 

0.91 

0.93 

1.00 

1.05 

1.10 

1.13 

1.19 

1.27 

NYC 

WDC 

0.91 

0.98 

1.00 

1.03 

1.13 

1.19 

1.26 

1.34 

1.38 

1.48 
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Origin 

Destination 

B11 

B12 

B13 

B14 

B15 

B16 

B17 

B18 

B19 

B20 

SUN 

ATL 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

SUN 

CHI 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.07 

0.07 

0.07 

SUN 

DEN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.02 

SUN 

HOU 

0.03 

0.04 

0.04 

0.04 

0.04 

0.05 

0.05 

0.05 

0.05 

0.06 

SUN 

IND 

0.01 

0.01 

0.01 

0.02 

0.02 

0.02 

0.02 

0.02 

0.02 

0.02 

SUN 

KSC 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

SUN 

LAX 

0.06 

0.06 

0.06 

0.07 

0.07 

0.07 

0.08 

0.08 

0.08 

0.09 

SUN 

NYC 

0.13 

0.14 

0.15 

0.15 

0.17 

0.18 

0.19 

0.20 

0.20 

0.21 

SUN 

SUN 

-0.32 

-0.34 

-0.36 

-0.38 

-0.41 

-0.44 

-0.46 

-0.48 

-0.49 

-0.52 

SUN 

SEA 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

SUN 

WDC 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.02 

0.02 

0.02 

SEA 

ATL 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.06 

0.07 

0.07 

SEA 

CHI 

0.28 

0.29 

0.31 

0.32 

0.34 

0.35 

0.37 

0.38 

0.41 

0.43 

SEA 

DEN 

0.05 

0.06 

0.06 

0.06 

0.07 

0.07 

0.08 

0.08 

0.09 

0.09 

SEA 

HOU 

0.21 

0.22 

0.24 

0.25 

0.26 

0.28 

0.30 

0.31 

0.32 

0.35 

SEA 

IND 

0.08 

0.08 

0.09 

0.09 

0.10 

0.10 

0.11 

0.11 

0.12 

0.13 

SEA 

KSC 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.06 

0.06 

0.07 

0.07 

SEA 

LAX 

0.35 

0.37 

0.39 

0.40 

0.41 

0.43 

0.45 

0.46 

0.49 

0.53 

SEA 

NYC 

0.81 

0.86 

0.91 

0.93 

1.00 

1.05 

1.10 

1.13 

1.19 

1.27 

SEA 

SUN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

SEA 

SEA 

-1.94 

-2.05 

-2.18 

-2.25 

-2.38 

-2.49 

-2.61 

-2.71 

-2.87 

-3.07 

SEA 

WDC 

0.06 

0.06 

0.07 

0.07 

0.08 

0.08 

0.08 

0.09 

0.09 

0.10 

WDC 

ATL 

0.05 

0.05 

0.06 

0.06 

0.06 

0.07 

0.07 

0.08 

0.08 

0.08 

WDC 

CHI 

0.31 

0.33 

0.34 

0.36 

0.39 

0.39 

0.42 

0.45 

0.47 

0.50 

WDC 

DEN 

0.06 

0.07 

0.07 

0.07 

0.08 

0.08 

0.09 

0.10 

0.10 

0.11 

WDC 

HOU 

0.23 

0.25 

0.27 

0.28 

0.30 

0.31 

0.34 

0.36 

0.37 

0.41 

WDC 

IND 

0.09 

0.09 

0.10 

0.10 

0.11 

0.11 

0.12 

0.13 

0.14 

0.15 

WDC 

KSC 

0.05 

0.06 

0.06 

0.06 

0.06 

0.06 

0.07 

0.07 

0.08 

0.09 

WDC 

LAX 

0.39 

0.42 

0.42 

0.44 

0.46 

0.49 

0.51 

0.55 

0.57 

0.62 

WDC 

NYC 

0.91 

0.98 

1.00 

1.03 

1.13 

1.19 

1.26 

1.34 

1.38 

1.48 

WDC 

SUN 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.01 

0.02 

0.02 

0.02 

WDC 

SEA 

0.06 

0.06 

0.07 

0.07 

0.08 

0.08 

0.08 

0.09 

0.09 

0.10 

WDC 

WDC 

-2.17 

-2.32 

-2.39 

-2.48 

-2.69 

-2.80 

-2.99 

-3.18 

-3.31 

-3.57 
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